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ABSTRACT 
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projects, group work, problem-solving tasks, reflective thought through 
writing, and a variety of other tasks that engage students in meaningful 
thinking. In addition, the report examines specific teacher beliefs 
reflecting a teaching philosophy consistent with the use of those activities, 
and beliefs reflecting a teaching philosophy that runs counter to these 
teaching practices. Data are presented in 43 tables throughout the report. 
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INTRODUCTION 



PERSPECTIVES ON TEACHING AND LEARNING 

No two teachers are perfectly alike in how they practice their craft and in their beliefs about teaching and 
learning that underlie those practices. Most teachers are eclectic, choosing from a large repertoire of 
teaching strategies as the particular situation warrants. However, behind all of these specific teaching 
practices and beliefs about teaching are two overarching approaches to teaching that represent different 
and somewhat incompatible models of good pedagogy: 

• Traditional Transmission Instruction is based on a theory of learning that suggests that students will 
learn facts, concepts, and understandings by absorbing the content of their teacher’s explanations or by 
reading explanations from a text and answering related questions. Skills (procedural knowledge) are 
mastered through guided and repetitive practice of each skill in sequence, in a systematic and highly 
prescribed fashion, and done largely independent of complex applications in which those skills might 
play some role. 

• Constructivist-Compatible Instruction is based on a theory of learning that suggests that understanding 
arises only through prolonged engagement of the learner in relating new ideas and explanations to the 
learner's own prior beliefs. A corollary of that assertion is that the capacity to employ procedural 
knowledge (skills) comes only from experience in working with concrete problems that provide 
experience in deciding how and when to call upon each of a diverse set of skills. 

The contrast between these two models has a long history in education, going back to debates about 
progressive education. However, current interest in constructivist-compatible instruction can be traced to 
the public and professional dialogues over alternative approaches to school reform that followed the 
publication of the Nation at Risk report (National Commission on Excellence in Education, 1983) — for 
example, the U.S. Department of Labor's proposal for emphasizing work-relevant competencies 
(Secretary's Commission on Achieving Necessary Skills, 1991). During the 1990's, various education 
reform efforts have proposed many different changes in school curricula, teachers' instructional strategies, 
and organizational arrangements (Hill and Celio, 1998). Many of these efforts have a strong constructivist 
flavor, with classroom activities, curricular content, and school organization all designed to be more 
consistent with a constructivist understanding of the learning process than are traditional practices and 
programs. (See, for example Lamon, et. al., 1996; Stringfield, Ross, and Smith, 1998). 

There are several basic aspects to the transmission-vs. -constructivist contrast. The basic difference, 
already introduced, is in terms of the theory of student learning that undergirds instructional practice — i.e., 
the difference between learning through reception of facts and repetitive practice of discrete skills versus 
learning through effortful integration of new ideas with those previously believed. 

A second critical element distinguishing transmission-based and constructivist-compatible instruction 
deals with the role of teacher and student . In a transmission-oriented learning theory, a teacher’s job 
consists of planning a set of activities in which particular subject content is experienced by students (e.g., 
teacher presentation of new information, students answering questions in textbook). Procedures for 
independent student work are defined in detail so that their work can efficiently be accomplished with as 
few errors and as little confusion as possible. In contrast, a constructivist learning theory maintains that 
because learning is a constructive process, mistakes and confusion sometimes provide the engagement that 
is needed for effortful learning. In constructivist teaching, the natural starting point for instruction is not 
the material to be taught, but student interests, prior experiences, and current understandings. Because not 
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all students begin at the same point, and have different questions they seek to answer, the content and 
activities of student work will vary within a classroom from student to student. The teacher’s role, in this 
approach, is to facilitate student-designed efforts. This is a more demanding job than merely studying and 
then presenting to students a pre-ordained body of content. Because constructivist approaches involve 
greater variety in the objects of learning across different students, and because they grant greater authority 
to students to decide upon the content of their learning, teachers implementing a constructivist-compatible 
pedagogy must have a wider range of knowledge and skills from which to draw. 

A third difference between these two pedagogical approaches is the importance that constructivism assigns 
to systematically created social structures for learning . Debates between students, cooperative group 
projects, and other activities involving the articulation of students' own ideas in concrete contexts are 
valued by constructivists for their power to further individual understanding. A transmission-oriented 
philosophy does not emphasize the social mediation of ideas because it sees understanding as coming from 
listening and reading — from receiving explanations directly — rather than as a result of actively working 
with and applying those ideas in a social context. 

In summary, a constructivist teaching approach attempts to make learning a more self-directed, personally- 
responsive, and socially-mediated process in which a learner's own motivation and effort are just as 
important, if not more central, to a student’ s education than the content or facts learned. This involves 
creating a learning environment so that students: 

• Identify their own issues and problems to be solved rather than having questions defined for them 

• Decide how to explore an issue or solve a problem rather than having these procedures defined by the 
teacher 

• Reflect further and makes sense of what they have experienced, and 

• Interact with peers by presenting their solutions, describing how solutions were reached, and receiving 
feedback. 

A classroom where these principles — rather than transmission- or skills-practice principles — guide a 
majority of moment-to-moment teaching decisions would be considered a “constructivist learning 
environment.” A constructivist learning environment tends to involve activities of the following five 
types: 

• Projects in which students employ a variety of skills and engage in a diverse set of tasks to accomplish 
a goal that, even if only implicitly, involves developing their understanding of important content. 

• Group work — where student tasks involve interdependencies with other students and, in particular, 
where discourse with other students is facilitated. 

• Problem-solving tasks: in other words, where the procedural knowledge present for solving a problem 
is not algorithmic but requires thinking, evaluating, decision-making, and planning as well; and where 
the definition of the problems themselves may be the responsibility of the student. 

• Reflective thought through writing . Exposition of a reasoned argument in written form is perhaps the 
most powerful and most general medium for engaging people with ideas — and with their development 
of understanding. 

• A variety of other tasks , in addition to reflective writing, that engage students in meaningful 
thinking — engaging them in a way that they consider both new information and their own prior 
understandings and beliefs and attempt to work out syntheses of both the old and the new. Those tasks 
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include, for example, having students make conjectures, eliciting their opinions, having them explicitly 
work on issues related to their own experiences, and arguing for various points of view. 

This report examines teachers’ survey responses that describe the frequency with which their teaching 
practice involves those five types of activities and the frequency with which their practice involves more 
traditional transmission and skill-development activities instead. In addition, the report examines specific 
teacher beliefs reflecting a teaching philosophy consistent with the use of these activities, and beliefs 
reflecting a teaching philosophy that runs counter to these teaching practices. 



THE STUDY OF TEACHER BELIEFS AND PRACTICES THROUGH SURVEY METHODS 

The policy community has not really reached a consensus about whether most teachers’ beliefs about good 
teaching as well as their actual frequency of employing different teaching practices is closer to a 
transmission theory or to a constructivist one. Some scholars (e.g., Cuban, 1993) argue that, influenced by 
strong cultural norms that support a conventional model of teaching and learning and by highly 
constraining work environments, most teachers hold beliefs consistent with a transmission pedagogy and 
only rarely engage their classes in constructivist-compatible activities. Others (e.g., Hirsch, 1996) argue 
that, influenced by colleges of teacher education where Deweyian world-views have predominated for 
many decades, most teachers, particularly elementary teachers, have a constructivist philosophy and 
(unfortunately, from Hirsch’ s viewpoint) implement that philosophy in their practice. Still others (Becker 
and Riel, 1999) argue that teachers’ philosophies are largely constructivist but that the school bureaucratic 
culture and public expectations for measurable documentation of student “achievement” severely constrain 
most teachers from implementing a constructivist pedagogy in daily practice. In sum, a variety of claims 
are made about the current distribution of teachers’ beliefs about good teaching and the frequency of 
specific teaching practices that are associated with those contrasting beliefs — both among all teachers and 
among different groups of teachers. 

Several recent national studies of teaching practice — e.g., the 1993 National Survey of Science and 
Mathematics education (Weiss et al, 1994), NCES’ 1994-95 Teacher Follow-up Survey (Henke, et al, 
1999), and the TIMSS international study (Schmidt, W. H. et al, 1997) — measured various aspects of 
teacher pedagogy. However, some analysts regard findings presented in reports of those studies with some 
degree of suspicion, in that only the TIMSS study was accompanied by studies of teacher practices by 
more in-depth observational methods (Schmidt, et al, 1996; and Stevenson, H. W. & Stigler, J. W., 1992), 
and none of the studies conducted any measurements of the validity of survey reports about teacher 
practices. 

Validity of Survey Measures of Teacher Beliefs and Practices 

Many researchers who observe teachers in day-to-day practice are quite suspicious of survey methodology 
for accurately measuring teacher beliefs or practices. To a large extent, survey questions can only get at 
the surface of teacher behavior whereas it seems plausible that important distinctions between effective 
and ineffective practices lie in the particular ways that teachers implement any particular "practice” such as 
"having students take greater responsibility in devising solutions to problems." Discourse practices, in 
particular, seem likely to be difficult to measure through survey approaches. Secondly, many people 
suspect that "social desirability" (i.e., survey respondents’ beliefs about how different responses would be 
evaluated by significant others) bias survey responses to questions about their beliefs and instructional 
practices. 

The most well-known study of the validity of survey measures of teachers’ behavioral self-reports is 
Burstein, et al ' s (1995) study of teachers' curricula and instructional practices. Their study was based on 
70 mathematics teachers from nine schools in two states, but rather than using actual classroom 





observations as validity criteria, the study used daily teacher logs and copies of teacher-made tests and 
homework assignments instead. Burstein, et al found that absolute agreement on most dimensions of 
mathematics teaching practice (e.g., how frequently teachers had students do such things as work with 
manipulatives, work individually on written assignments, or work in small groups) was generally in the 
40% to 60% range, using 4-point scales. As Mayer (1999) argues, that level of agreement is not 
impressive, given there were only four response points to the survey items. 

The Burstein study, though, looked at validity in only one of several possible ways. Two dimensions of 
validity measurement need to be considered: (1) "absolute" agreement versus "relative validity," the 
relative ordering among respondents; and (2) the validity of individual survey items versus that of an index 
or compilation of items. 

Absolute agreement is the similarity between survey reports and criterion measures — for example, how 
frequently teachers used manipulatives to demonstrate a concept. Relative validity is the correlation across 
teachers in their relative placement along each of two measures, a survey measure and a criterion (e.g., 
observational) measure. Absolute agreement, the focus of the Burstein study, is required for accurate 
descriptions of teaching practice across a population of teachers. However, similar relative placements are 
all that is required in an analytic study to confirm that variations in teacher pedagogy are associated with 
variations in other aspects of teaching — for example, subject taught or the teacher’s use of computers. 

The second distinction is between the validity of individual survey questions and the validity of indices 
built by scaling or factor analyzing a number of indicators of a common underlying dimension. The 
Burstein study was concerned with the validity of individual survey indicators rather than indices. Yet, as 
psychology students are taught from their first class in statistics, single dichotomies, or even 4- or 5-point 
response items, inevitably suffer from limited precision, and in fact represent only indicators of more 
general constructs, which are inherently of greater interest. 

An earlier study, by Smithson and Porter (1994), provided correlational validity measurements for 
individual survey items, such as teachers’ use of lab work, small group work, and recitation activities. 
Generally, Smithson and Porter found correlations between frequencies as reported on teacher surveys and 
frequency of entries in teacher logs in the 0.4 to 0.6 range for nearly all of the (fairly general) instructional 
practices they examined. Mayer himself conducted another study (Mayer, 1999), albeit a small one, of nine 
Algebra teachers in one district, where actual classroom observations were employed as the criterion 
measure. He found that, despite very low over-time (4-month) reliability measures of 17 individual 
pedagogical practices (e.g., make conjectures, engage in student-led discussion, work from a textbook, 
take computational tests), an index-level survey measure was highly correlated (+0.85) with a 
corresponding measure based on substantial classroom observations (three periods spaced over three 
weeks). The index was based on the relative frequency of 13 reform practices compared to the frequency 
of 4 traditional practices. Thus, although absolute individual-level findings may suffer from unreliability 
and bias compared to more in-depth measurements, relative validities may be quite satisfactory for many 
purposes (i.e., differences between contrasting teacher populations), and index-level correlational analysis 
may have very high validity. 
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Preliminary Validity Study 

Not every survey measure of teacher belief or teaching practice may be equally valid or reliable. The 
Smithson-Porter study found correlations as low as 0.21 (“write a report or paper”) and as high as 0.65 
(“do lab or field work”). In preparation for our national survey, Computer Technology and Instructional 
Reform, which is the source of the data on teacher beliefs and practices in this report, we conducted a 
validity study in order to select survey items that were most highly correlated with observation and 
interview-based criteria of similar dimensions of belief and practice. 

The validity study was conducted with 72 teachers from 24 schools in three parts of the country. In 
contrast to Mayer’s study, our teachers were heterogeneous in subject responsibilities and school level. 
Like Mayer, we correlated teacher survey responses with coded observation data. However, we also 
interviewed respondents at length (roughly 2 hours per teacher), examined written assignments, quizzes, 
and other available artifacts, and had interviewer-observers complete the same survey instrument as the 
teacher respondents as if they were an “objective observer” of that respondent’s practices and teaching 
philosophy. In addition to survey items asking the teacher to generalize about her use or frequency of use 
of different teaching practices — for example, “When you ask students questions, how frequently are you 
doing so in order to. . . get students to explain their reasoning?” — the validation survey asked questions 
about teaching philosophy as well — for example, agreement or disagreement with the suggestion that 
students must acquire basic skills before they are ready for “meaningful” learning. 1 

In general, we found that although most single-item survey measures correlated only in the 0.20 to 0.40 
range with single-item criterion measures (Becker and Anderson, 1998), when we combined several of the 
better-correlating and substantively related survey items into an index and correlated the index with more 
complex validation criteria (factor scores based on multiple sources of evidence), the relationships were 
substantially stronger. For the 24 aspects of constructivist philosophy and practice having at least three 
survey items that had correlated at least 0.30 with the objective-observer scoring of that item, the median 
index-to-factor correlation was 0.51. 2 When we built three second-level indices out of more 
comprehensive sets of survey items (representing the three primary dimensions of constructivism in our 
thinking at the time — meaningful learning tasks, cognitively demanding tasks, and social learning), the 
correlations with similarly defined factor scores were, respectively, 0.60, 0.62, and 0.68 (Becker, 2000b). 3 

Overall, the use of indices and focusing on a "relative" rather than "absolute" perspective on validity made 
us quite confident that self-report measures of pedagogy and philosophy would be sufficiently valid for the 
primary analytic purposes of our national survey. 



1 “Validation” in our preliminary study meant three things: (1) the correlation of survey items with the parallel 
“objective observer” item scored by the research team member; (2) the correlation of individual survey items with 
"validation factor scores" built by combining a set of coded responses made by the research team members from a 
variety of evidence (interviews, observations, coding of artifacts, and the "objective observer" report); and (3) the 
correlation of an index combining substantively related teacher survey responses with the validation factor scores 
described above. The indices and validation factor scores represented efforts to operationalize theoretically derived 
components and sub-components of the overall construct of constructivist-compatible pedagogy. Items measuring 
substantively similar beliefs and practices were combined in the analysis because exploratory factor analysis found no 
way to consistently distinguish them based on factor loadings. (In the current study, our factor analyses did 
consistently distinguish between belief and practice survey items. See Becker and Anderson (1998) and Becker 
(2000b) for more details about the validation study. 

2 The median number of items in an index was 5. The range was from 3 to 10 items per index. 

3 There were 20 survey items measuring meaningful tasks, 23 measuring cognitively demanding tasks, and 19 
measuring social learning. 
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For the national survey, we selected items based on their validity but also in order to accomplish a broad 
domain coverage of the constructivist-transmission-orientation distinction. In addition, we modified items 
and restructured some in order to optimize domain coverage in minimum space and with attention to the 
particular items that correlated the best with our criterion measure. (For example, we combined pairs of 
items that had presented contrasting perspectives into several paired comparisons.) For the items drawn 
from the validity study that were used in the national study, an index of the 21 items measuring meaningful 
learning tasks correlated 0.54 with the factor score for that component of the validation items (i.e., 
interview, observation, and other items measuring the same aspects of the construct); the index of the 26 
items measuring cognitively demanding tasks correlated 0.70 with its validation factor score, and the index 
of 1 1 items measuring social learning correlated 0.67 with its factor score. The lower correlation for the 
meaningful-tasks component of constructivist pedagogy appears to be due to the greater heterogeneity of 
that construct. 



THE TEACHING, LEARNING, AND COMPUTING 1998 NATIONAL SURVEY 

Our data on teachers’ beliefs about teaching and learning and their teaching practices comes from a 
national survey, conducted during the Spring of 1998, that focused on the relationship between teacher 
pedagogy and their use of computers in teaching. The study was funded by the National Science 
Foundation with additional funds provided by the Office of Educational Research and Improvement, U.S. 
Department of Education. Called Teaching, Learning, and Computing — 1998 (TLC), the survey involved 
questionnaire responses from three different samples of schools — a national probability sample, a 
purposively drawn sample of schools with the greatest per-capita computer technology, and a purposively 
drawn sample of schools that were participating in some way in substantial educational reform efforts, 
either self-initiated or as part of one of 50 different national or regional reform programs. Within each 
school, the TLC survey collected data from the principal, the school technology coordinator, and a 
probability sample of teachers. The teachers were selected in a way that disproportionately oversampled 
those who made substantial use of computers, who had students do project work, and who emphasized 
higher-order thinking in their teaching. However, all analyses use weights that compensate for differential 
sampling rates of different types of teachers so that the results can be seen as coming from a representative 
sample of teachers at the schools surveyed. For the most part, the information in this report comes from 
the national probability sample of schools; however, the more analytic work incorporates the purposive 
samples as well, and when that is done it is noted in the text or accompanying table. 

The teachers whose responses are described in this report number 4,083, including 2,251 in the national 
probability sample of schools. The teachers are those who taught classrooms from the 4 th through the 12 th 
grades in all subjects except physical education and special education. The teachers completed 20-page 
survey instruments. Some survey questions were asked to subsets of respondents (four different 
overlapping questionnaire versions were used), so some of the data was not based on the full sample 
surveyed. The teacher respondents represent 69% of the teachers sampled; and the schools participating 
include 75% of the schools initially selected. Additional information regarding the methodology of the 
study is provided in Appendix B. 
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PART I. TEACHERS’ BELIEFS ABOUT TEACHING AND LEARNING 

To understand teachers’ pedagogy, it is important to understand their beliefs about what constitutes good 
instructional practice. Presumably, their own instructional practices reflect, to a large extent, what they 
believe to be good teaching, and their beliefs about good teaching reflect their understandings about how 
students learn. Of course, many other factors affect teachers’ individual practices besides their 
philosophies of teaching — for example, class sizes that are larger or more heterogeneous than they 
expected, the content of the specific textbook provided to them, resources which they expected but which 
were not provided, explicit directives from administrators, influences of their peers, etc. An important 
element in this project’ s investigation into teachers’ use of computers is to inquire whether teachers who 
use computers frequently with students have become more constructivist in practice than their own 
teaching philosophy would suggest that they would be (and to study under what circumstances, those 
effects take place). To do that, we must know to what extent teaching philosophy does predict 
instructional practice, and preparatory to doing that , we examine the character and distribution of specific 
beliefs that make up those teaching philosophies. 



DESCRIPTIVE FINDINGS 

Teachers’ beliefs about good teaching, and more implicitly, their beliefs about the nature of learning, were 
measured mainly through three questions in the TLC survey: one question which presented a paragraph- 
length vignette describing how two hypothetical teachers — Ms. Hill and Mr. Jones — characteristically 
taught their class; a second question in which several pairs of contrasting statements of teaching 
philosophy were presented, asking respondents to choose on a 5-point scale which statement among the 
pair came closest to their own point of view; and a set of general statements about teaching and learning, 
presenting respondents with 6 alternatives from “strongly disagree” to “strongly agree.” The full range of 
responses by teachers in the national probability sample to each of the items in these three questions, as 
well as to the questions measuring actual teacher classroom practices, is provided in Appendix C. 

The “Ms. Hill vs. Mr. Jones” vignettes were used to gauge teachers' overall preference for, and beliefs 
about, contrasting direct instruction and constructivist instructional styles. (See Figure 1 for the vignettes 
used.) Although most teachers can see value and a reason to teach like either teacher, in different 
situations, a preference for Mr. Jones’ pedagogy suggests a clearer belief in the value of constructivist 
instructional reforms. Respondents were asked to evaluate the two alternative teaching styles with respect 
to four criteria: with which approach were they more comfortable, which approach did they think students 
preferred, from which approach did they think students gained more knowledge, and from which approach 
did they think students gained more useful skills. 



FIGURE 1: VIGNETTES DESCRIBING CONTRASTING INSTRUCTIONAL STYLES 



Ms. Hill was leading her class in an animated 
way, asking questions that the students could 
answer quickly; based on the reading they had 
done the day before. After this review, Ms. Hill 
taught the class new material, again using simple 
questions to keep students attentive and listening 
to what she said. 



Mr. Jones’ class was also having a discussion, but 
many of the questions came from the students 
themselves. Though Mr. Jones could clarify 
students’ questions and suggest where the 
students could find relevant information, he 
couldn’t really answer most of the questions 
himself. 



Overall, more teachers felt comfortable with (64%) and thought students preferred (53%) the traditional 
style of Ms. Hill, with fewer selecting Mr. Jones. (The middle or “undecided” position was chosen by 
many teachers as well; see Table 1.) Moving quickly over content may pose fewer problems for teachers 
and students and therefore seem easier. However, in terms of the consequences for students, teachers were 
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more likely to believe that Mr. Jones’ approach was better. Concerning students gaining more knowledge 
from Hill or Jones, teachers were evenly split — with more than 40% favoring each approach. 
Concerning the acquisition of useful skills many more teachers favored Mr. Jones’ approach (57% 
favoring Jones, 29% favoring Hill). This suggests that teachers think students will benefit from some use 
of inquiry-oriented teaching that places more responsibility on students. At the same time, they recognize 
that it is difficult to carry out many of those practices, particularly because not all students are eager to 
participate in classroom learning organized around those practices. 

TABLE 1: TEACHERS’ BELIEFS ABOUT CONTRASTING INSTRUCTIONAL STYLES 



Favored Ms. Hill Middle position on Favored Mr. Jones 
(traditional approach) 5-point scale (constructivist approach) 



They would be more comfortable 


64% 


7% 


28% 


They think students prefer 


53% 


10% 


37% 


Students gain more knowledge 


44% 


14% 


42% 


Students gain more useful skills 


29% 


14% 


57% 



Sample: Probability sample, questionnaire versions 3 and 4. 



In the paired comparison question, five pairs of philosophical positions were presented, four of which we 
discuss here. One expressed the contrast in the role of the teacher between being a facilitator of student 
learning versus an explainer of material to students, very much encapsulating the difference in approach 
between Ms. Hill and Mr. Jones in the earlier question. A second pair contrasted a teaching approach 
where multiple activities were going on in class at the same time, activities suggestive of complex project 
work and a fair amount of latitude for students, versus a classroom where everyone was working on the 
same assignment, one with “clear directions, and. . .that can be done in short intervals that match students’ 
attention spans and the daily class schedule.” A third comparison was between the position that 
curriculum content coverage was more important than “encouraging sense making" among students versus 
the opposite position, and the fourth was whether promoting student interest or learning content skills was 
more important. (See Figure 2.) 



FIGURE 2: PAIRED COMPARISONS MEASURING CONTRASTING TEACHER BELIEFS 



Constructivist Perspective 




Traditional Transmission Perspective 


“1 mainly see my role as a facilitator. 1 try to provide 
opportunities and resources for my students to 
discover or construct concepts for themselves.” 


Vs. 


"That's all nice, but students really won't leam the 
subject unless you go over the material in a structured 
way. It's my job to explain, to show students how to 
do the work, and to assign specific practice." 


"It is a good idea to have all sorts of activities going 
on in the classroom. Some students might produce a 
scene from a play they read. Others might create a 
miniature version of the set. It’s hard to get the 
logistics right, but the successes are so much more 
important than the failures." 


Vs. 


"It's more practical to give the whole class the same 
assignment, one that has clear directions, and one 
that can be done in short intervals that match 
students' attention spans and the daily class 
schedule." 


"The most important part of instruction is that it 
encourage “sense-making” or thinking among 
students. Content is secondary.” 


Vs. 


"The most important part of instruction is the content 
of the curriculum. That content is the community’s 
judgment about what children need to be able to know 
and do." 


"It is critical for students to become interested in 
doing academic work — interest and effort are more 
important than the particular subject-matter they are 
working on." 


Vs. 


"While student motivation is certainly useful, it should 
not drive what students study. It is more important that 
students learn the history, science, math and 
language skills in their textbooks." 



Overall, teachers are substantially more constructivist than traditional in their responses to each of these 
items. Twice as many teachers agreed that there should be multiple project-oriented activities going on as 
favored short-duration whole-class assignments instead. Even more teachers believed that their 
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instructional planning should focus on constructing meaning and on student interest than on coverage of 
curriculum and textbook content (by margins of 2.5 to 1 and 3 to 1 respectively; see Table 2). The 
facilitator-explainer contrast received a much more balanced reaction, reflecting the ambivalence teachers 
reported regarding these contrasting approaches in their responses about Ms. Hill and Mr. Jones from the 
prior question. 



TABLE 2: TEACHERS’ AGREEMENT WITH CONTRASTING STATEMENTS OF TEACHING PHILOSOPHY 





Favored the More 
Constructivist 
Statement 


Middle position on 5- 
point scale 


Favored the More 
Traditional Position 


Facilitator vs. Explainer 

Multiple project activities vs. short-term 


40% 


30% 


30% 


whole-class assignments 


48% 


26% 


26% 


Sense-making vs. Curriculum coverage 


49% 


31% 


19% 


Student interest vs. Textbook content 


55% 


27% 


18% 



Sample: All teachers in probability sample. 



These findings suggest that given a brief argument made between support for a philosophical position 
consistent with constructivist instructional reform and one reflecting a more traditional viewpoint, many 
more teachers will select agreement with reform than with traditional teaching practice. 

Even when presented with statements solely supporting traditional teaching practice (without a paired 
argument favoring the opposite position), a majority of teachers typically will reject the practice. This is 
seen by the teachers’ responses to the third survey question probing their beliefs about good teaching 
practice. Eight of ten “agree-disagree” questions posed statements consistent with traditional teaching 
philosophy (shown in Table 3). Of the eight, a majority of teachers disagreed with five of them and almost 
a majority (48 to 49%) disagreed with two others. A huge majority of teachers (85%) rejected the idea that 
because teachers know more than students, they “shouldn’t let students muddle around when they can just 
explain the answers directly.” Even more teachers (91%) rejected the idea that student projects aren’t 
useful because they “often result in students learning all sorts of wrong ‘knowledge.’” More than three- 
fifths of the teachers (62%) rejected the idea that “instruction should be built around problems with clear, 
correct answers, and around ideas that most students can grasp quickly” — a clear statement of rejection of 
the principles of direct instruction formulated two decades ago and around which most traditional 
instruction is based. (See Table 3.) In contrast, none of the statements reflecting traditional educational 
philosophy received anywhere near the level of endorsement as did one of the two statements in this 
survey question that reflected a constructivist philosophy: that “students should help establish criteria on 
which their work will be assessed.” More than two-thirds of responding teachers endorsed that principle 
(71%), but the maximum agreement for any of the eight “traditional” statements was only 58% — a 
statement about homework being a good setting for answering questions in the textbook. 
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TABLE 3: PERCENT OF AGREEMENT AND DISAGREEMENT 
WITH OPINION STATEMENTS ABOUT PEDAGOGY 



Traditional Pedagogy 

Student projects often result in students learning ail sorts of wrong "knowledge" 
Teachers know a lot more than students; they shouldn't let students muddle around 
when they can just explain the answers directly 

Instruction should be built around problems with clear, correct answers, and around 
ideas that most students can grasp quickly 
A quiet classroom is generally needed for effective learning 

Students are not ready for "meaningful" learning until they have acquired basic reading 
and math skills 

How much students leam depends on how much background knowledge they have — 
that is why teaching facts is so necessary 

It is better when the teacher — not the students — decides what activities are to be 
done 

Homework is a good setting for having students answer questions posed in their 
textbooks 

Constructivist Pedagogy 

Students should help establish criteria on which their work will be assessed 
Students will take more initiative to leam when they feel free to move around the room 
during class 



Percent Percent 

Agreeing Disagreeing 



9 


91 


16 


84 


38 


62 


41 


59 


46 


54 


51 


49 


53 


47 


58 


42 


71 


29 


52 


48 



Note: Each statement had three levels of agreement and three levels of disagreement, which are combined for this summary. 
Sample: Probability sample. Each statement except the two constructivist ones was asked of only three-quarters of the teachers, 
balanced among the four questionnaires. 



Regardless of whether these statements of teaching philosophy carry over to teachers’ actual teaching 
practice, their endorsement of ideas about good teaching consistent with constructivist instructional reform 
and their equally strong rejection of traditional ideas about teaching suggest one of two things: Either 
teachers believe that cultural norms are suddenly supporting reform practices (and that they felt they 
should endorse those norms) or these beliefs are actually ones that the teachers hold. Although we do not 
discount the possibility that social desirability plays some role in the distribution of survey responses (and 
that includes the possibility that respondents perceived researcher opinion to favor constructivist 
statements which social desirability led them to endorse), still the consistent choices of teachers throughout 
these three survey questions favoring constructivist philosophies cannot be that easily dismissed. Our 
conclusions: as a whole, U.S. 4 th through 12 th grade teachers believe in a much more constructivist basis 
for teaching than they are often given credit for. 



LEVEL AND SUBJECT-MATTER DIFFERENCES IN TEACHING PHILOSOPHY 

There are certainly many reasons why some teachers hold philosophies of teaching more compatible with 
constructivist theories of learning than other teachers do. Their differing personal and educational 
backgrounds, different prior experiences in teaching, and different current teaching responsibilities all play 
a role. In this report, we will explore some of the clear differences in teaching philosophy between 
teachers teaching at different school levels and having different subject-matter responsibilities. 



Elementary Teachers. Particularly Those Who Teach Self-Contained Classes, Have More 
Constructivist Teaching Philosophies 

When we compare upper elementary teachers (i.e., those who teach grades 4 to 6 in elementary schools) to 
teachers at the middle school and high school level, it is clear that elementary teachers are more often 
constructivist in their philosophies, and they agree with “traditional” views of good teaching less often 
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than secondary teachers do. For example, when asked whether students are ready for “meaningful” 
learning before they have acquired basic reading and math skills, most secondary teachers (51%) say 
“no” — they say essentially that reading and math skills must be mastered first. However, nearly three- 
fourths of elementary teachers (71%) believe that meaningful learning can co-occur with the development 
of basic reading and math skills. Elementary teachers are also more likely to be comfortable with Mr. 
Jones’ tum-back-students’ -questions approach to teaching than Ms. Hill’s direct instruction approach. 

They are also more likely to believe in having multiple project-oriented activities going on in class than 
teaching short-attention-span whole-class assignments. However, on most of our teacher belief questions 
the differences between elementary and secondary teachers are under 10 percentage points. 

Why elementary teachers should be somewhat more constructivist in philosophy is probably partly due to 
their having a greater interest in child development per se and a lesser attachment to the knowledge base of 
particular subject matters. That is, the average elementary teacher may be more attentive to the 
mechanisms that produce learning and less interested in the transmission of their own knowledge than the 
average secondary teacher. Another reason may be related to structural differences in their work 
environment. Most elementary teachers are responsible for one group of students for most of a school day. 
In these self-contained classes they have a greater opportunity for flexibility in their teaching. To help 
clarify this issue, we split elementary teachers into those who indicated they taught self-contained classes, 
versus those who identified their teaching responsibilities as subject-specialists or having responsibility for 
two or three subjects taught to different classes of students. 

We found that self-contained elementary teachers do give more constructivist-compatible responses to 
questions about their teaching beliefs than other elementary teachers do. For example, more self-contained 
elementary teachers favored Mr. Jones’ inquiry-based approach (59%) than did other elementary teachers 
(47%). This pattern was also evident when we asked about the utility of different forms of assessment in 
judging how well students are learning. (See below.) Among elementary teachers of self-contained 
classes, 73% saw open-ended problems as being a very useful assessment tool while only 54% of other 
elementary teachers did. Teachers of self-contained classes were also somewhat less likely than other 
elementary teachers to believe that “a quiet classroom is generally needed for effective learning” (39% vs. 
48%). Finally, when confronted with having to choose between covering standard curriculum content or 
instruction that encourages critical thinking and “sense-making,” elementary subject-specialists chose 
curriculum content nearly twice as often as elementary teachers of self-contained classes (24% vs. 13%). 

Differences Among Secondary Teachers by Subject Taught 

Among secondary school teachers, there are a wide variety of beliefs about good teaching. Even within 
subjects, opinions vary. However, subject matter is among the most important predictors of teachers’ 
opinions. 

To illustrate these differences, we will show the pattern for two specific survey items: whether the teacher 
believes that students learn more knowledge from Mr. Jones’ inquiry approach or Ms. Hill’s direct 
instruction approach; and whether they see their role more as a facilitator for student discovery or more as 
the explainer (“Students really won’t learn the subject unless you go over the material in a structured 
way.”) These two survey prompts deal with the same basic difference in approach between a teacher 
helping students to discover knowledge for themselves and a teacher who transmits that knowledge 
directly to students. When one explains something, to some extent it means that all the meaningful 
questions have been answered — and that transmission of these answers should be the goal of instruction. 
However, if one believes that one cannot deeply understand something through simply receiving an 
explanation, one is more likely to value a teaching style that is more inquiry-oriented and constructivist. 
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Table 4 shows how secondary teachers of different subjects answered those two survey items. Regarding 
the question about the Mr. Jones’ inquiry approach and Ms. Hill’s direct instruction style, four groups of 
teachers were more likely to say that learning would be greater if students were given responsibility for 
investigating their own questions: English teachers, fine arts teachers, science teachers and teachers of 
computer classes (e.g., “Computer Applications” or “Introduction to Computers”). Teachers of the other 
five subjects were more likely to believe that rapid-fire delivery of questions with single correct answers is 
a better approach to helping students learn their subject: mathematics, social studies, foreign language, 
business education, and vocational education teachers. These subject differences were not small. In seven 
of the nine subjects, one or another of these views was clearly dominant (i.e., led the other by more than 10 
percentage points). 

On the more general survey item about the same issue — do the teachers see themselves as a facilitator of 
student work or person who explains what students need to know — the pattern was fairly similar except 
that vocational education teachers switched to the “constructivist side” and two other groups (social studies 
teachers and business education teachers) were evenly split. (One difference between the questions is that 
in the vignettes, Jones’ class involved specifically intellectual discourse whereas in this second item the 
facilitation was merely of “students discovering. . .for themselves.”) Still, even for this second item, 
teachers of two subjects, mathematics and foreign language were much more likely to see their teaching in 
terms of explaining things to students than in terms of facilitating student work. 

TABLE 4: SECONDARY TEACHERS 1 RESPONSES TO TWO SURVEY ITEMS ABOUT CONSTRUCTIVIST 
VERSUS TRANSMISSION MODELS FOR APPROACHING TEACHING, BY SUBJECT MOST OFTEN TAUGHT 



Students Gain More Knowledge from Mainly See my Role as a 
Jones 1 or Hill’s approach Facilitator or Explainer 

(remainder were undecided) (remainder were undecided) 



Subjects Sorted by Constructivism of 
Responses to These Two Questions 


% Jones 1 Inquiry 
Approach 


% Hill’s Direct 
Instruction 


% Facilitator 


% Explainer 


Computer 


47 


39 


56 


24 


English 


52 


38 


46 


23 


Fine Arts 


43 


30 


42 


28 


Science 


45 


32 


37 


32 


Vocational 


30 


50 


47 


30 


Business 


36 


52 


38 


37 


Foreign Language 


43 


47 


24 


39 


Social Studies 


28 


55 


34 


37 


Math 


34 


49 


28 


43 



Sample: Both probability and purposive samples. Jones-Hill question from questionnaire versions 3 and 4 only. 



BELIEF INDEX SCORES 

Exploratory factor analysis suggested that, like the two survey prompts just discussed, most of the items in 
the three survey questions about teaching philosophy could be interpreted as indicators of a common 
underlying construct that contrasts constructivist versus transmission and skills-practice orientations. 
Reliability analysis of 13 of these items produced an index with an alpha of 0.83. The following are these 
13 survey items, which are expressed in constructivist terms when both poles were presented to 
respondents, and marked as “reversed” when the prompt asked the respondent to “agree or disagree” with 
a transmission-oriented statement. 

• Jones’ inquiry approach produces more student knowledge than Hill’s direct instruction 

• Jones’ inquiry approach produces more student skills than Hill’s direct instruction 

• Believes in being a facilitator rather than explainer 

• Student interest and effort is more important than textbook content 

• Sense-making and thinking are more important than the specific curriculum content 
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• Different students engaged in different project-type activities is better than the whole class working at 
the same time on a series of short-duration assignments 

• Students will take more initiative if they are free to move around the room 

• Students should help establish the criteria on which their work is assessed 

• Instruction should be built problems with clear, correct answers (reversed) 

• Teachers know more than students and shouldn’t let students muddle around (reversed) 

• Student learning depends on background knowledge — that’s why teaching facts is so necessary 
(reversed) 

• It is better for the teacher, not students, to decide what activities are to be done (reversed) 

• A quiet classroom is generally needed for effective learning (reversed) 

Overall, across these 13 items, teachers of elementary grades are somewhat more constructivist in 
philosophy than other teachers are (particularly those who teach self-contained classes), and high school 
teachers are somewhat less constructivist than other teachers. By subject matter, two groups of secondary 
teachers have more constructivist philosophies than do others: English teachers and a relatively small 
group of primarily middle-grades teachers who teach a variety of academic subjects. (See Table 5.) Three 
groups of secondary teachers are generally less constructivist than are others: social studies teachers, math 
teachers, and foreign language teachers. Teachers of those subjects more often agree with the "traditional” 
transmission-oriented views of instruction. (Secondary science teachers tend to be squarely in the middle.) 
Among non-academic or "applied" subjects, computer teachers had views slightly on the constructivist 
side, while the business education and fine arts teachers in our national sample are clearly transmission- 
oriented in their beliefs. 



TABLE 5: AVERAGE TEACHER BELIEF INDEX Z-SCORES BY SUBJECT AND LEVEL 





Elementary 

Grades 


Middle 

Grades 


High 

School Grades 


Both Secondary 
Levels 


Secondary "Academic" 


English 


- 


0.08 


0.47 | 


0.27 


Science 


- 


0.23 


-0.13 


0.02 


Math 


- 


- 0.19 


-0.40 


-0.29 


Social Studies 


- 


-0.44 


-0.17 


-0.31 


Foreign Language 


- 


-0.72 


-0.25 


-0.46 


Mixed & Other Academic Secondary 


- 


0.52 


-0.34 


0.40 


Secondary "Applied" 










Computers 


- 


1 0.37 


| -0.18 


0.08 


Other Secondary Applied 


- 


-0.16 


0.20 


0.06 


Vocational 


- 


-0.24 


-0.06 


-0.12 


Business 


- 


-0.48 


-0.21 


-0.25 


Fine Arts 


- 


- 0.56 


| -0.01 i 


| - 0.28 


Elementary 










"Self-contained" 


0.17 


- 


- 


- 


"Other" 


0.10 


- 


- 




Ail Subjects 


0.15 


-0.02 


-0.07 


-0.05 



Note. Z-scores across all teachers in probability sample (Mean = 0.00, s.d. = 1.00). Cells for which a relatively high mean z- 
score minus one standard error of that mean remains above +0.20 and cells whose (relatively low) mean z-score plus one 
standard error of that mean remains below -0.20 are highlighted in the table. Additionally, where the difference between high 
school and middle school means is greater than the sum of their standard errors, a box is drawn around the level with the more 
constructivist beliefs. 



Although, on average, middle school teachers have slightly more constructivist beliefs than high school 
teachers do, the patterns are quite different for different subjects. For some subjects, notably those of the 



more qualitative sort, high school teachers are more constructivist. For example, it is primarily high 
school English teachers, not middle school, who hold strongly constructivist beliefs — among the strongest 
of any group in our sample. Furthermore, in several of the subjects where transmission-oriented views 
predominate — social studies, foreign language, and fine arts — it is at the middle school level where 
transmission-oriented views are particularly strong; high school teachers of those subjects are much closer 
to the middle of the teaching philosophy distribution. On the other hand, in the more technical and 
quantitative subjects of science, mathematics, and computer education, middle school teachers are more 
inquiry-oriented (constructivist) than their high school teacher peers. In fact, high school math teachers 
are more transmission-oriented than any other group of high school teachers in our national sample. 

How would we account for these different patterns of belief by teachers in different subjects, and why do 
such contrasting patterns prevail between middle and high school teachers in different subjects? To 
explain subject-matter differences, and to some extent differences in level as well, one has to consider the 
nature of the curriculum in the different subjects. In areas (subjects and levels) where the most salient 
competencies are conceptual ones, teachers are more apt to see student learning as arising from 
engagement in issues in which the student has a personal interest. Conceptual learning seems to demand 
an approach that facilitates the emergence of student understanding from their continued intellectual 
engagement with the subject. Conversely, in areas where the most salient competencies are defined as 
involving procedural and factual knowledge, teachers may be more inclined towards direct explanation and 
an emphasis on “clear, correct answers.” Of course every field of study could be approached from either a 
conceptual or procedural direction. However, every field has a characteristically normative set of beliefs 
about the appropriate timing of attention to procedural knowledge and conceptual understanding. 
Differences in teaching philosophies across fields, and across levels within fields, may largely come from 
parallel differences in those curriculum-based beliefs. 

In addition, differences in philosophies within a subject between middle grade and high school teachers 
may also be due to other factors that operate differentially across subjects. Older students’ more 
developed abilities to reason independently and critically about complex issues might lead high school 
teachers to believe more in the kind of teaching that provides latitude to students; however, such views are 
likely to prevail only in those subject areas where reasoning is valued relative to skills or knowledge 
acquisition. On the other hand, high school teachers of some subjects may feel a greater sense of 
ownership in the knowledge base that exists in their field (or a greater sense of pressure to prepare their 
students for college admission), and may therefore feel a greater desire to directly convey that knowledge 
base to their students. Overall, then, the specific strength of these forces — internal or external pressure to 
cover curriculum versus expectations for more independent thinking by students — may explain why for 
some subjects, teachers’ philosophies appear to be more constructivist among middle grades teachers while 
for other subjects, more constructivist philosophies prevail among high school teachers. 

We will return to these issues later in this report, after we examine in detail specific instructional practices 
of teachers of different subjects, and the extent to which those classroom practices reflect a constructivist 
or a transmission-oriented pedagogy. However, before discussing teachers’ practices, we need to examine 
one other area regarding teachers’ beliefs — their judgments about the usefulness of different forms of 
assessment in deciding how well students are learning. 
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BELIEFS ABOUT METHODS OF ASSESSING STUDENT LEARNING 



Teachers choose from an array of methods to evaluate their students’ acquisition of competencies and 
knowledge. These methods range from brief self-constructed short-answer quizzes and externally 
designed (i.e., “standardized”) multiple-choice tests all the way to small group and individual research 
projects several weeks in the making. A teacher’s choice of assessment methods is one way in which she 
implements a teaching philosophy, and constitutes one element of her actual classroom practice. However, 
rather than asking teachers about their actual assessment practices, the TLC survey asked teachers to 
evaluate the usefulness of different methods of assessing student learning. Those judgments constitute 
another aspect of teaching philosophy . 

Given the widespread use in schools of short-answer and multiple-choice tests (including those provided to 
teachers by textbook publishers), and given the widespread discussion in the policy community about the 
use of externally designed standardized multiple-choice tests for accountability purposes, it is important to 
note that most teachers do not find those approaches as “useful... in judging how well students are 
learning” as others in their repertoire. Overall, among the six assessment methods the teachers were asked 
to rate in terms of usefulness for themselves, the three judged most useful were student presentations and 
performances, individual and group projects, and open-ended problems. By far the least useful method, in 
the view of teachers, was standardized test results, and the next least useful was “short-answer and 
multiple-choice tests.” In fact, in not a single academic subject did teachers say that standardized tests, 
short answer, or multiple choice assessments were among the most useful assessments available to them. 

In every subject, project work, student performances, and open-ended problems were viewed as being 
more useful than either short-answer/multiple-choice tests or standardized test results. (See Table 6.) 

TABLE 6: PERCENT OF TEACHERS REPORTING EACH KIND OF ASSESSMENT 
IS “VERY USEFUL IN JUDGING HOW WELL STUDENTS ARE LEARNING.” 



Middle and High School Grades 





Elementary English 


Social 

Studies 


Science 


Math 


Other 

Secondary 


Total 


Student presentations and performances 


75 


75 


69 


59 


53 


80 


71 


Open-ended problems 


67 


74 


58 


65 


60 


68 


66 


Individual and group projects 


74 


64 


54 


61 


44 


72 


65 


Essay tests 


58 


79 


69 


55 


28 


54 


56 


Short-answer and multiple-choice tests 


41 


23 


44 


38 


40 


34 


36 


Standardized test results 


14 


13 


18 


9 


22 


14 


15 



Sample: Probability sample. 

As one would expect, there is an association between valuing student presentations, projects, and open- 
ended problem-solving and constructivist teaching philosophies; and, conversely, valuing short- 
answer/multiple choice tests and standardized tests and a transmission-oriented philosophy. (See Table 7.) 



TABLE 7: CORRELATIONS BETWEEN BELIEF INDEX AND USEFULNESS 
OF DIFFERENT KINDS OF ASSESSMENT 



Student presentations/performances 


.36 


Individual and group projects 


.36 


Open-ended problems 


.33 


Essay tests 


.06 


Standardized test results 


-.20 


Short-answer and multiple-choice 


-.35 



Sample: Probability sample. 



O 

EKLC 



15 



w' 



18 



These findings suggest that, as a group and in every subject, a large majority of teachers believe that if the 
purpose of assessing students is to find out how well students are learning, then assessment should be 
based on providing students with opportunities to demonstrate and communicate their understandings and 
ability to solve problems rather than on the basis of seeing which answers they select from a list or what 
factual information they can recall. This result reinforces the position that teachers have primarily 
constructivist viewpoints about teaching and learning. 

Critics of standardized testing and advocates of alternative assessment strategies should be encouraged by 
these findings. They illustrate that, to a large extent, teachers also question the utility not only of 
standardized tests, but even the short-answer and multiple-choice style tests that textbook publishers 
provide to them and that they create themselves, constrained by limited time and the need to have some 
measurement on the 150 students they teach. For people who are concerned about testing policies and that 
element of the reform” movement based on the idea of keeping teachers and students accountable through 
standardized test comparisons, it is important to recognize that teachers themselves seem to be fairly 
cognizant of the limitations of the kind of testing practices that such an accountability system would 
promulgate. 
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PART II. CONSTRUCTIVIST-COMPATIBLE TEACHING PRACTICES 



TEACHING PRACTICES AND TEACHING PHILOSOPHY 

In the previous section, we examined a number of beliefs about teaching and learning that together reflect 
a philosophy, which we argue, is compatible with constructivist learning theory. However, carrying out a 
teaching practice based totally on such beliefs is anything but easy to accomplish. Even when teachers 
have constructivist beliefs, limitations coming from (a) the strain of being responsible for entire 
classrooms of individual students at the same time, (b) inadequacies in their own knowledge of content or 
pedagogy, (c) competing objectives that they may have themselves, or (d) external pressures on what they 
must do in the classroom — all of these forces interfere with the implementation of a constructivist 
philosophy. What is implemented, based on what is relatively easy to implement and what draws less 
interference from competing forces, often bears only a superficial resemblance to the philosophy that a 
teacher may espouse. 

For example, a fairly consistent theme in the literature on school improvement is the notion that broad 
coverage of a large number of topics does not permit students to engage a given issue deeply enough to 
generate a web of connected and meaningful understandings. If constructivist-believing teachers have 
other objectives that include, for example, broad coverage of diverse content (whether personally valued or 
because of outside pressures), then those teachers will inevitably fail to be as constructivist in practice as in 
philosophy. In fact, one question in the survey asked teachers to characterize the number of topics 
(themes, units, chapters) that they cover in one class — the class in which they actually feel they most often 
achieve their teaching objectives. Only 7% of teachers surveyed reported covering a small number of 
topics in great depth in that class, while one-half (actually, at least 50% in every subject) reported covering 
a large or very large number of topics without going into very much depth. 



CONSTRUCTIVIST-COMPATIBLE TEACHING PRACTICES 

The central element of constructivist learning theory is that learning occurs when a student is deeply 
engaged with an intellectual issue. According to this theory, learning occurs only when students make an 
effort to construct their own understanding out of a problem situation. Moreover, student effort occurs 
primarily around learning tasks that students regard as meaningful. That is, although teachers can get 
students to do schoolwork by using various sanctions and incentives, constructivism maintains that deep 
understanding only occurs if students are motivated by a desire to understand and that that motivation is 
much more likely if (a) she already has an interest in and some knowledge of the content and (b) if the 
content is made "accessible" to the student by somehow making explicit connections to the student's 
current perspective or understandings. Teaching that organizes student work around meaningful activities 
so that students are thoughtfully engaged with content is the heart of a constructivist teaching practice. 

Thoughtful engagement with content would be expected, for example, if students are required to monitor 
their own learning experience as part of their class work. Examples of meaningful self-monitoring would 
include writing about their own work, considering multiple potentially valid alternatives, helping to plan 
classroom activities, designing their own problems to solve, or undertaking tasks where there is not a clear 
“correct” answer ahead of time. Other examples of teaching practices which suggest that students are 
thoughtfully engaged in intellectual work include frequent writing assignments of a reflective nature, 
having students make conjectures to explain things and backing up those conjectures with reasoning and 
evidence, and expressing the same idea in multiple forms. These are among the operational indicators we 
use to define a constructivist-compatible teaching practice. 
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Other teaching practices are also compatible with constructivism but do not appear to be inevitably 
associated with a constructivist philosophy — practices such as having students work in groups, do 
laboratory activities, or work on complex projects on a single topic. That is, we do not automatically 
conclude that teachers’ use of these practices means that their principal intent is that students be 
thoughtfully engaged with content in the same way that we are willing to assume that thoughtful 
engagement is the goal of reflective writing assignments or asking students to make conjectures. Projects 
and small group work can be assigned without attention to deep intellectual engagement; they are activities 
that teachers can, to some extent, be expected to report whether or not they are particularly constructivist. 
In sum, they may be assigned and carried out without being intellectually challenging. 

The question of the intellectual engagement occurring in these common “constructivist-compatible” 
activities is therefore important to address. In what settings do projects and group- work provide a vehicle 
for meaningful student engagement? 

Thus, although our study describes a variety of teaching practices associated with a constructivist- 
compatible teaching practice, we define certain practices — the ones most directly linked with student 
intellectual engagement (meaningful thinking) — as central, while other practices — notably the use of 
projects and group activities — we define as probabilistically indicative of a constructivist practice. The 
following section discusses 27 teaching practices — how they are conceptualized, how they are related to 
one another, and how they are distributed across school levels and subjects. The section is divided into 
five main groups of survey items: (1) indicators of “meaningful thinking” teaching practices including 
reflective writing; (2) indicators of “problem-solving”; (3) indicators of teaching using “group-work”; (4) 
indicators of a “project-based" approach to teaching; and (5) indicators of a traditional, transmission- and 
skills-oriented teaching practice. 

Each indicator (survey item) is uniquely assigned 4 to one of these categories, based on a large number of 
exploratory factor analytic investigations. 5 We find that our particular categorization of the 27 indicators 
is useful and defensible statistically, but we make no claim that this typology and our assignment of survey 
items has more than heuristic value. Figure 3 lists each of the 27 survey items discussed in this section, 
according to the category to which we assigned it. Appendix D presents the distribution of responses by 
teachers in the national probability sample to each of the items in the five survey questions that were used 
to develop the dimensions of teaching practice examined and the overall Constructivist Pedagogy Index. 

One other point. When we compare teachers of different subjects on these indices of instructional practice, 
as we do below, we sometimes find substantial differences in the average scores of teachers in different 
subjects. Although we are aware that specific instructional practices are differentially relevant to the 
teaching of different subjects, we believe that, for the most part, differences across teaching subjects in 
average index scores reflect real differences in the constructivism of teaching practices experienced by 



4 One survey item, "work in small groups to come up with a joint solution or approach to a problem or task," is 
included in both the "group-work" and "problem-solving" measures. 

5 Survey items that had low communalities during initial factor analyses were excluded for subsequent analyses. We 
did not use factor scores in this study, but used factor analysis as a technique for decisions about item inclusion in 
indices. We used oblique rotation within a principal axis factoring method because we envisioned the different types 
of practice, while distinct, to be nonetheless related and therefore not orthogonal constructs. Belief items, when they 
were included, almost always loaded on separate factors from practice items, in contrast to the findings of the 
preliminary validation study. In general, it was during this stage that we observed that project-related items were least 
well correlated with the other practice items; i.e., always producing their own factor. In contrast, reflective writing 
and the other meaningful thinking items appeared to be very closely related. Group- work items tended to load with 
the meaningful thinking items, pointing to a possible contrast between projects and group work in how each relates to 
the level of cognitive challenge in a classroom. 
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students in classes in these subjects. At the same time, differences in mean scores by subject may also 
reflect the particular survey items selected to measure each aspect of instructional practice. Without 
further investigation, it is difficult to know which of these two explanations is more appropriate for any 
given comparison. Consequently, as a conservative approach, in this report we pay more attention to 
differences among teachers of the same subject than to the overall differences among teachers across all 
subjects. 
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FIGURE 3: SURVEY ITEMS MEASURING CONSTRUCTIVIST PRACTICE 
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COGNITIVELY CHALLENGING TEACHING PRACTICES 



Three groups of survey items comprise our measure of the degree of emphasis placed by a teacher 
on providing a cognitively challenging experience for students. Of course, what is cognitively 
challenging for one student may be routine and even boring for another. However, as distinct 
from emphasizing skill- and knowledge -transmission activities, some teachers emphasized 
reflective writing, tasks requiring deep thinking (such as asking students probing questions), and 
problem-solving tasks. While in some factor analyses, problem-solving items loaded on a 
different statistical factor from the others, the items classified as reflective writing and those 
called "deep thinking" differed primarily in terms of the content of the activity. Together, the 
reflective writing and deep thinking items are referred to here as "meaningful thinking" teaching 
practices. 



Reflective Writing 

Perhaps the most powerful indicator of meaningful intellectual engagement is writing in which 
students are expected to draw connections between ideas themselves. The three types of writing 
assignments we asked about were writing in journals, writing of reflective essays, and writing an 
analysis of the quality of their own work. 6 Teachers who engage their students in these reflective 
writing activities, regardless of subject, also tend to assign other types of meaningful tasks. What 
types of reflective writing activities are reported across subjects, and which type of writing is 
most common? 

Of the three types of reflective writing, the most commonly reported was writing essays in which 
students “are expected to explain their thinking or reasoning at some length.” One half of all 
teachers, across all subjects, reported assigning essays of this type on at least a monthly basis, and 
nearly one-quarter (22%) reported assigning reflective essays on a weekly basis. Fewer teachers 
had students do any journal writing at all, but of those who did, more teachers had students do 
journal- writing on a weekly basis (28% of all teachers). Written self-assessment was by far the 
least frequent of these three methods. 

Not surprisingly, reflective writing assignments are much more common in English classes than 
in other subjects. In fact, in the secondary grades, more than a third of all essay and journal 
writing that was reported took place in English classes. (See Appendix Table A-l.) Aside from 
English classes, the secondary subjects where essays are most likely to be assigned are science 
and social studies. Elementary teachers of self-contained classes also report frequent journal 
writing by students. 7 For example, 58% of elementary self-contained and 51% of English 
teachers have students write in a journal at least once per week, compared to only 14 to 15% of 
science and social studies teachers. Among applied subjects, vocational teachers appear to be 
more likely to have students do journal writing than in other subjects; 26% report students 
keeping a monthly journal compared to only 5% of business teachers. However, there are only 
small differences in the other types of writing in these subjects. 



6 The self-assessment item also incorporated “serious discussion” among students as well as written 
assessments. 

7 Of course, teachers of self-contained classes have their students in class for many more hours per week 
(and are the primary source of their students' home assignments). 
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In some subjects, there were large differences between middle and high school teachers’ use of 
reflective writing assignments. Middle school teachers reported substantially more writing 
activities in science, computer education and mathematics. For example, 34% of middle school 
science teachers said students wrote in a journal compared to 9% of high school science teachers. 
In applied subjects, the pattern was the same. Almost half of middle school computer teachers 
said they had students write essays monthly (47%) compared to 13% of high school computer 
teachers. (See Appendix Table A-2.) 



TABLE 8: PREVALENCE OF REFLECTIVE WRITING ACTIVITIES BY SUBJECT TAUGHT 





Percent Reporting 
Monthly Writing by Students 




Rank of Subjects by Reflective Writing Scores 


Essay 
Explaining 
Thoughts at 
Length 


Write in 
Journal 


Essay or 
Discussion 
Seriously 
Assessing 
Own Work 


Reflective Writing 
mean 
Z-score 


English 


82 


59 


54 


0.71 


Elementary Self-contained 


64 


65 


42 


0.59 


Elementary Other 


44 


37 


33 


0.05 


Social Studies 


67 


21 


35 


0.02 


Science 


49 


20 


21 


-0.18 


Foreign Language (HS)* 


47 


17 


44 


- 0.32 


Vocational 


25 


26 


20 


- 0.45 


Computers 


30 


16 


21 


-0.51 


Business (HS) 


31 


4 


28 


-0.62 


Math 


24 


18 


10 


-0.66 


Fine arts 


15 


9 


27 


-0.68 


All Teachers 


50 


35 


31 





Sample: Probability sample. 
*HS = high school. 



Tasks and Questions Calling for Deep Thinking 

While reflective writing is clearly one way of promoting meaningful thinking around an activity, 
many other approaches are used, particularly in subjects where writing is less a primary objective 
of the subject matter than in English. Additional activities we view as providing an indication of 
meaningful student engagement include (a) the types of questions teachers ask, (b) how they 
introduce new topics (e.g., by having students make conjectures), and (c) how classroom time is 
spent (e.g., in student-led vs. teacher-led discussions). 8 

Questions without Answers: 

One key question we can ask about the intellectual nature of classroom activity is whether or not 
student work is built around questions or tasks for which the teacher has a clear, pre-ordained 
“correct” answer in mind. Where objectively knowable "right answers" prevail, students who are 
making an effort may be focused on finding out the teacher's answer ("what am I supposed to 




8 Many people would regard teacher-led discussion as no less likely to promote meaningful thinking than 
student-led discussion. This may be the case, although for the students who are presenting to their peers, 
the requirement to communicate with an audience would seem to make greater thoughtfulness more likely 
for them. Overall, in our data, frequency of student-led discussions is positively associated with other 
elements of cognitive challenge, such as reflective writing and problem-solving, whereas frequent teacher- 
led discussions is negatively correlated with those elements. 

26 



22 



learn?”) and be less inclined towards achieving a personal mastery and deep understanding of the 
larger content under study ("what do I think or understand about this issue?"). Teachers of 
different subjects vary quite a bit in terms of the nature of the questions they ask students and the 
ambiguity of what counts as knowledge, but overall relatively few teachers reported having 
frequent discussions where the knowledge at issue involved much ambiguity. When asked how 
often how they give assignments "where there is no indisputably correct answer — where truth is 
complex and perhaps impossible to know,” only 7% of teachers said they did this on a weekly 
basis and only 23% said they did this at least monthly. The subject where teachers were least 
likely to teach in this way was business education (none weekly and only 10% monthly) and 
mathematics (1% weekly; 11% monthly). In comparison, about 40% of English and fine arts 
teachers reported organizing student work around tasks where truth was inherently ambiguous. 

A similar indicator that most teachers’ pedagogies are structured around presenting truths rather 
than students constructing their individual understandings comes from a question about how 
teachers introduce a unit of instruction. We asked teachers whether, at the outset of a unit, they 
raised questions to which they themselves did not know the answer, an approach that might allow 
them to model the development of understanding. Do many teachers present themselves as 
learners, raising problematic questions with respect to their own knowledge? Overall, fewer than 
one-quarter of teachers said that in introducing their current unit, they had brought up questions to 
which they didn’t know the answer. In particular, very few business education teachers did this 
(5%), far fewer than did teachers in other fields. 

An additional note about the validity of this survey’s data on teaching practices. As discussed 
earlier in the report, writers such as Cuban (1993) and Kennedy (1999) have questioned whether 
teachers’ survey responses to general questions about their instruction accurately portrays the 
frequency that they carry out particular teaching practices. Cuban, in particular, argues that in- 
depth observation of teachers in classrooms yields a conclusion that typical practice is much more 
traditional and fact- and skill-oriented than is suggested by teachers’ survey responses. However, 
we find that teachers' responses vary a great deal according to specific practices that they are 
asked about. The findings of the preceding paragraphs, in particular, suggest that very few 
teachers report employing approaches that seem fairly central to a constructivist practice — giving 
students tasks related to problems that lack clear, correct answers and modeling the role of learner 
and by admitting to ignorance of important issues in an upcoming curricular unit. The distribution 
of answers to these two survey questions, and others with similarly "traditional" modal responses 
is evidence that teachers’ survey self-reports may be a fairly honest reflection of their teaching 
practice — and that despite having fairly constructivist beliefs about what constitutes good 
teaching, most teachers employ only some teaching strategies in pursuit of a constructivist 
teaching philosophy. 

Other Questions Teachers Ask of Students: 

Virtually all teachers can be expected to spend some time leading whole-class discussions in 
which they ask students questions of various sorts. But the nature of whole-class discussion is 
open to variation. Here we examine a few different types of questions teachers ask groups of 
students and what their goals are in asking students questions. We asked how often teachers are 
trying to accomplish the following goals — seeing if students know the correct answer; seeing if 
they have done homework; eliciting student ideas or opinions; getting students to justify or 
explain their reasoning; and, having students relate what they are working on to their own 
experiences. 
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All teachers at least occasionally ask students questions to see if they know the correct answer, 
and half of the respondents in our survey said they did this "very often" or "always." Fewer 
teachers reported asking students questions to see if homework had been done; however, a clear 
majority of foreign language teachers and almost exactly half of business education teachers said 
they asked questions to check homework "very often." Although this highlights the traditional 
approach of these teachers, across all subjects the most frequently reported reasons for asking 
questions are those that would help provide the teacher with insights into their students' 
thinking — going beyond just whether the students know the right answer. 

Asking questions that elicit students' own ideas is one of the more common, and probably less 
difficult, ways of engendering interest and thoughtfulness in a class of students. A large majority 
of teachers reported that they "very often" asked students questions in order to elicit their ideas. 
Similarly, most teachers said they asked questions to get students to justify their reasoning, and a 
majority of teachers said they asked questions that asked students to relate ideas to their own 
experiences. (See Table 9.) More teachers reported asking those kinds of questions than said they 
asked questions in order to see if students knew the correct answer or to see if they had done their 
homework. However, compared to other subjects, math and business education teachers were less 
likely to ask questions in order to elicit student ideas and opinions. Perhaps teachers in those 
subjects more often feel that students do not have the requisite knowledge to have formed 
"useful" opinions, compared to English and social studies, where students have more personal 
experience in the subject which they can draw upon. 

TABLE 9: REASONS TEACHERS GIVE FOR ASKING STUDENTS QUESTIONS, BY SUBJECT TAUGHT 

Percent Asking Question for this Purpose Very Often or Always 

Have students 



Ask for the 
correct answer 


Check if 
students did 
homework 


Have students 
justify or explain 
their reasoning 


Elicit student 
ideas and 
opinion 


relate work to 
their own 
experience 


English 


42 


27 


80 


76 


68 


Social Studies 


52 


33 


65 


70 


55 


Science 


48 


30 


77 


61 


54 


Math 


66 


39 


88 


55 


37 


Foreign Language 


75 


57 


53 


73 


53 


Computers 


54 


27 


56 


59 


48 


Business 


71 


49 


73 


53 


50 


Vocational 


42 


26 


65 


61 


62 


Fine arts 


46 


16 


63 


65 


49 


Elem. Self-contained 


53 


33 


81 


78 


69 


Total 


53 


32 


76 


69 


57 



Sample: Probability sample. 

When we look at middle school versus high school patterns of question-asking by subject, in 
some cases it is the high school teachers who ask for more probing questions; in some cases, it is 
the middle school teachers. High school English teachers generally appear to be more 
constructivist than middle school teachers in their question-asking practices — much less 
frequently looking for correct answers (29% of high school vs. 52% of middle school teachers 
saying they did this "very often") and more often asking students to justify or explain then- 
reasoning. This pattern may be due to the greater focus on skills instruction in middle school 
English and a greater focus on literature in high school. (See Appendix Table A-3.) 

Similarly, among social studies teachers, high school teachers pursued more probing questions 
than did their middle school counterparts. Three-quarters of social studies teachers at the high 
school level (76%) reported asking students to justify or explain their reasoning very often, 
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compared to only 57% of middle school social studies teachers. More than three-fifths at the high 
school level (62%) asked students to relate work to their own experience, compared to 49% of 
middle school social studies teachers. Thus, at least in this respect, high school social studies and 
English teachers appear to be more constructivist than middle school teachers of the same 
subjects. 

In mathematics, the pattern is reversed. Middle school math teachers were more likely to report 
having their students justify or explain their reasoning than were high school math teachers (93% 
vs. 80%) and more likely to say they had students relate their work to the students' own 
experiences "very often" (47% vs. 26%; see Appendix Table A-3.) 

The survey asked teachers about one other questioning strategy. We asked respondents whether, 
in introducing a new unit to students, they questioned students in order to get them to make 
conjectures about the topic of the unit. This is an approach that can activate students' existing 
related knowledge and opinions and can uncover student interests and preconceptions about the 
topic. Overall, half of all teachers in our study indicated that they asked students to make 
conjectures when they introduced the current unit. About two-thirds of elementary and English 
teachers reported doing this. However, teachers of some other academic subjects — specifically 
mathematics and social studies teachers — were among the least likely to ask students to do this 
kind of preliminary thinking and hypothesizing. 

Several other survey items reflecting practices that prioritize meaningful thinking were included 
in the survey: for example, holding debates where students argued from viewpoints that may not 
be their own, having students address the same issue in different ways (multiple representations), 
and having students themselves plan classroom activities. Each of these tends to place 
responsibility for creating knowledge in the hands of students. Holding debates where students 
argue a point of view that may not be their own forces students to view different sides of an issue 
and to use evidence in support of taking a position. Only 12% of teachers overall reported using 
this practice even on a monthly basis; the most frequent affirmative responses were given by 
social studies teachers (24%). Use of multiple representations was reported by about one-half of 
teachers in each of the academic subjects, but only between one-quarter and one-third of 
“applied” teachers. Students "helped to plan learning activities" in about one-quarter of all their 
classes, most often in English (38%) and least often in science (17%), math (12%) and business 
(10%) classes. 

Index of Meaningful Thinking 

In every school subject there is a high correlation among reflective writing tasks, teachers asking 
probing questions, and other tasks requiring relatively deep thinking. That is, within a subject the 
teachers who do more reflective writing also report more of the other thinking tasks as well, and 
vice versa. Taking each subject separately, the correlation between the reflective writing index 
(comprised of three items) and an index comprised of 10 other deep thinking tasks (see below) is 
greater than 0.45 for elementary, social studies, English, and science teachers. (The correlation 
coefficients are presented in Appendix Table A-4.) Because of this pattern, we created a 
combined measure of “meaningful thinking” that is based on teachers' scores on reflective writing 
and on the other tasks and questions calling for deep thinking that we asked about. Combining 
these items for use as a criterion measure was strongly supported by empirical analysis — e.g., the 
items consistently loaded together in factor analysis. The alpha reliability of the scale, for 
example, was 0.81. Corrected item-total correlations were all in the range of 0.30 to 0.60. More 
importantly, we think that, conceptually , an index of these items makes sense, and, as suggested 
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earlier, we regard an emphasis on meaningful thinking as the central aspect of constructivist- 
compatible teaching practice. 



TABLE 10: FREQUENCY OF MEANINGFUL THINKING BY SUBJECT AND LEVEL TAUGHT 

(INDEX MEAN Z-SCORE) 



Ranking by 
Subject & Level* 


Mean Z-Score on Meaningful 
Thinking Index (13 items) 


English (HS) 


0.68 


English (MS) 


0.45 


Elementary 


0.34 


Social Studies (HS) 


0.19 


Science (MS) 


-0.03 


Social Studies (MS) 


-0.16 


Vocational (All) 


-0.26 


Science (HS) 


-0.27 


Foreign Language (All) 


-0.27 


Math (MS) 


-0.32 


Computer (All) 


-0.50 


Fine Arts (All) 


-0.66 


Math (HS) 


-0.66 


Business (All) 


-0.71 


All Teachers 


0.01 



Sample: Probability sample. 

*MS = middle school, HS = high school. 



Overall, English teachers appear more likely to assign work that would generate meaningful 
thinking than any other group of secondary teachers. Only elementary teachers approach English 
teachers in this respect. However, elementary teachers' scores are difficult to compare with 
secondary teachers' because most of the items are "frequency -based" (i.e., estimates of how often 
different practices occur) and most elementary teachers teach the same students for a full day, far 
longer than secondary teachers who generally teach classes for only 50-minute periods. 

Three of the four groups of teachers who report the least frequent use of these reflective writing, 
probing questions, and deep thinking tasks teach subjects that are applied in nature rather than 
strictly academic — computer teachers, business education teachers, and fine arts teachers. 
However, high school math teachers also fall in this group, and even when we use a broader 
definition of higher-order thinking that incorporates problem-solving (see below), the typical high 
school math teacher still appears to put less of an emphasis on cognitive engagement than other 
teachers do. 

Problem-Solving Activities 

Problem-solving involves a more-or-less cognitively challenging activity that can also be more or 
less meaningful. In other words, the construct of problem-solving could include students solving 
routine algorithmic problems in a workbook. However, the particular aspects of problem-solving 
which are included in the TLC questionnaires — students working on problems with no obvious 
solution, deciding on their own problems to solve, and deciding on their own procedures to try — 
clearly are cognitively challenging. We expected that teachers who reported more of these types 
of problem-solving activities also would report more of the other forms of meaningful student 
engagement. 
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Not many teachers report that they have students do this kind of challenging problem-solving 
very often. Only a minority of teachers — fewer than two out of five — report that they have 
students work on problems with no obvious solution even on a monthly basis. About the same 
proportion say they have students develop their own procedures for solving a problem and then 
discuss what they did with their peers. Fewer teachers, only one-fifth, have students design their 
own problems to solve, as often as once per month. These statistics suggest that most student 
work in classrooms is closely prescribed, focused on factual knowledge or specific skills, and is 
teacher-designed. 

Generally, we would expect math teachers to be among those who report the most involvement in 
problem-solving activities. However, this is true only in one respect: math teachers are more 
likely than other teachers to report having students decide on their own procedures for solving a 
problem and then discuss results with their classmates. On the other types of problem-solving 
(working on problems with no obvious method of solution; and solving problems by their own 
procedures), math teachers were not distinctly different from science, English, and social studies 
teachers. The one exception is that social studies teachers much less often reported having 
students design their own problems to solve (only 8%, combining middle and high school 
teachers). 

For subjects where there are a sufficient number of teachers at both middle and high school 
levels, Table 11 presents information about subject-specific use of problem-solving strategies by 
level. Middle school math teachers are more likely than high school teachers to have students 
work on problems with no obvious method of solution (42% vs. 31%), and are also more likely to 
have students decide on their own procedures and discuss results among themselves (57% vs. 
40%). For vocational teachers, the pattern was reversed, with high school teachers reporting 
more problem-solving activities. 



TABLE 11: FREQUENCY OF PROBLEM-SOLVING ACTIVITIES BY SUBJECT AND LEVEL TAUGHT 

Percent of teachers reporting students at least monthly... 

Work on problems Mean Z-Score on 



Ranking by 
Subject & Level* 


with no obvious 
method of solution 


Design their own 
problems to solve 


Decide on their own 
procedures & discuss 


Problem-Solving 
Index (4 items) 


Math (MS) 


42 


24 


57 


0.33 


Vocational (All) 


28 


42 


47 


0.27 


Elementary 


40 


27 


46 


0.25 


Science (MS) 


34 


21 


36 


0.14 


Math (HS) 


31 


18 


40 


0.03 


Computers (All) 


34 


37 


54 


0.02 


English (HS) 


42 


21 


35 


-0.01 


Science (HS) 


36 


20 


32 


-0.09 


Social Studies (HS) 


46 


6 


26 


-0.17 


English (MS) 


26 


11 


26 


-0.27 


Fine Arts (All) 


37 


22 


21 


-0.32 


Social Studies (MS) 


26 


11 


35 


-0.51 


Foreign Language (All) 


10 


17 


9 


-0.58 


Business (All) 


9 


10 


23 


-0.78 


All Teachers 


35 


21 


38 


0.01 



Sample: Probability sample. 

*MS = middle school, HS = high school. 
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Table 1 1 also has a summary column showing the average score of teachers in each level-subject 
combination on an overall Problem-Solving Index. 9 Overall, middle school mathematics, 
secondary vocational, and elementary teachers report the highest average scores while middle 
school social studies, secondary foreign language, and business education teachers have the 
lowest average scores on Problem-Solving. 

Problem-Solving and Use of other Meaningful Thinking Strategies 

Although in our exploratory factor analyses, the problem-solving items generally loaded on a 
separate factor from the other items labeled Meaningful Thinking, teachers who use problem- 
solving strategies are much more likely to also use other thinking strategies as well. Table 12 
provides data, by subject and level, on the extent to which users and non-users of each problem- 
solving activity differ in terms of their use of meaningful thinking strategies. The measure used 
in this and subsequent tables is the effect size (E.S.) — the difference in mean scores between two 
groups of teachers on the criterion measure (in this case, the teacher's Meaningful Thinking 
score), in standard deviation units. 10 In addition to comparing users and non-users of specific 
problem-solving activities, Table 12 provides comparisons between teachers who were above and 
below the median on an index that combines frequency-of-use data on all four problem-solving 
activities, including one not shown in the table. The effect size statistic for the overall problem- 
solving index measures how much teachers who are above the median on problem solving also 
report engaging students in meaningful thinking activities compared to other teachers of the same 
subject and level who do less problem-solving. 11 

Table 12 shows that in every subject, the teachers who report more problem-solving activities are 
much more likely to report engaging students in meaningful thinking. The effect size is higher 
than +0.6 for every subject, and greater than +0.9 for the major academic subjects. For example, 
the middle school math teachers who are above the median on Problem Solving are 1 .0 standard 
deviations higher on Meaningful Thinking than the middle school math teachers below the 
median on Problem Solving. That translates into a difference of roughly 40 percentile points. All 
three problem-solving items shown (students deciding own procedures, problems with no obvious 
solution, designing their own problems to solve) had equally large effect sizes on meaningful 
thinking. 12 




9 The Problem-Solving index, like others developed here, combines scores on component survey items 
(four items in this case, including one not shown in the Table), by standardizing each component item 
before taking the mean across items. The transformation to z-scores shown similarly maintains a constant 
level of variability across indices for purposes of comparing across different measures. 

10 The particular Meaningful Thinking score used is the within-subject z-score, defined so that the average 
score for every subject is set at 0 with a standard deviation of 1 . 

Another way of presenting the same information is through correlation coefficients between Problem- 
Solving and Meaningful Thinking. Those are shown in Appendix Table A-4. 

Another interpretation of the positive correlation between Meaningful Thinking scores and Problem- 
Solving scores (or between any pairs of measures that are based on "frequency" estimates or similar Likert 
scales is that the association is a function of different questionnaire response styles among teachers. That 
is, some teachers are more likely to give higher frequency estimates than other teachers are, regardless of 
the item prompt. Such response styles no doubt play a role. However, the size of these associations — most 
of those reported involve effect sizes above 0.5 — suggest that more than response styles are at work here. 
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TABLE 12: EFFECT SIZE OF PROBLEM-SOLVING ON MEANINGFUL THINKING 



Meaningful Thinking E.S. by Problem-Solving Practice Meaningful Thinking E.S. 
(at least monthly use vs. non-users) by Problem-Solving 



Ranking by 
Subject & Level* 


Design their own 
problems to solve 


Decide on their 
own procedures 
& discuss 


Work on problems 
with no obvious 
method of solution 


Index 

(Top 50% on index 
vs. bottom 50%) 


Math (MS) 


1.1 


0.9 


0.7 


1.0 


Vocational (All) 


0.5 


1.0 


0.4 


0.6 


Elementary 


0.8 


0.7 


0.8 


0.9 


Science (MS) 


0.9 


1.0 


0.9 


1.1 


Computer (MS) 


0.5 


0.5 


0.6 


0.7 


Math (HS) 


1.2 


0.8 


0.7 


0.9 


Computers (All) 


0.7 


0.9 


0.4 


0.9 


English (HS) 


0.9 


1.0 


1.0 


1.1 


Computer (HS) 


0.9 


1.2 


0.5 


1.1 


Science (HS) 


0.7 


1.0 


0.9 


1.0 


Social Studies (HS) 


1.2 


1.2 


1.1 


1.1 


English (MS) 


0.6 


0.9 


0.9 


1.0 


Fine Arts (All) 


0.9 


1.1 


1.0 


1.2 


Social Studies (MS) 


1.6 


0.8 


1.2 


1.1 


Foreign Language (All) 


0.6 


0.6 


0.7 


0.6 


Business (All) 


0.9 


0.9 


0.5 


1.1 


All Teachers 


0.8 


0.8 


0.9 


1.0 



Sample: Probability and purposive samples. 
*MS = middle school, HS = high school. 



The pattern across the first three columns in Table 12 suggests the issues around which 
meaningful thinking may be most likely to occur in different subjects. For example, in middle 
school social studies, teachers who report giving their students the task of designing their own 
"problem to solve" had average scores on the Meaningful Thinking index that were 1.6 standard 
deviations higher than teachers who did not use that approach. This suggests that meaningfulness 
in middle school social studies revolves around students choosing topics for their work. In 
vocational classes, on the other hand, teachers’ scores on Meaningful Thinking were particularly 
high for teachers who reported having students decide on their own procedures. This suggests that 
meaningful thinking in vocational classes may be centered more around procedural activities. 

In summary, within every subject, teachers who report doing more problem-solving also appear 
to engage students in more meaningful thinking. The combination of problem-solving and 
meaningful tasks provides a cognitively challenging classroom environment. In the remainder of 
this report, we employ an index of Cognitive Challenge which combines the information from the 
13 survey items concerning meaningful thinking and 3 of the 4 items that asked about problem- 
solving. 13 Because the Cognitive Challenge Index is central to the rest of this report, we present, 
in Table 13, the average Cognitive Challenge scores by subject and level. Those scores closely 
match the data on Meaningful Thinking presented in Table 10, except that the high Problem- 
Solving scores of middle school math teachers and vocational education teachers raise then- 
average Cognitive Challenge scores closer to the midpoint for all teachers. In contrast, scores of 
English and social studies teachers are somewhat reduced for this measure of Cognitive 
Challenge compared to the prior measure. Again, the score for elementary teachers is not strictly 



13 The fourth Problem Solving item, "working in groups to jointly solve a problem," was excluded from the 
combined Cognitive Challenge index because it was assigned instead to the "Group Work" component of 
constructivist practice. 
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comparable to the others because the vast majority of such teachers teach all-day self-contained 
classes and many of the survey items are affected by the overall extent of teacher contact with the 
same students. 



TABLE 13: FREQUENCY OF COGNITIVE CHALLENGE BY SUBJECT AND LEVEL TAUGHT 

(INDEX MEAN Z-SCORE) 



Ranking by 


Mean Z-Score on Cognitive 


Subject & Level* 


Challenge Index (16 items) 


English (HS) 


0.53 


Elementary 


0.33 


English (MS) 


0.27 


Social Studies (HS) 


0.09 


Science (MS) 


-0.01 


Vocational (All) 


-0.14 


Math (MS) 


-0.17 


Science (HS) 


-0.25 


Social Studies (MS) 


-0.29 


Computer (All) 


-0.36 


Foreign Language (All) 


-0.45 


Math (HS) 


-0.54 


Fine Arts (All) 


-0.60 


Business (All) 


-0.82 


All Teachers 


0.01 



Sample: Probability sample. 

*MS = middle school, HS = high school. 



ACTIVE LEARNING STRATEGIES: PROJECTS AND WORK IN SMALL GROUPS 

Discussions of reform teaching practices often focus on students working actively on "authentic" 
projects and often involve both collaborative work groups and a specific end-product developed 
for a specific audience. However, we have introduced constructivist pedagogy in a completely 
different way — focusing on instruction that is cognitively challenging as a result of the teacher 
designing tasks that are personally meaningful to students. The obvious research questions are 
whether these two aspects of teaching practice — active, collaborative project-based work and 
cognitively challenging tasks — (a) tend to occur among the same teachers and (b) tend to occur at 
similar relative frequencies across the various secondary subjects and levels. In other words, are 
the teachers who frequently have students do project and group work the same teachers who 
provide cognitively challenging instruction? Are the subjects and levels where we find higher 
rates of cognitively challenging instruction the same subjects and levels where higher-than- 
average rates of project and group work activities prevail? 

One can easily imagine that many teachers whose classes do projects and group assignments do 
so for reasons other than a belief in a constructivist learning theory. They may use projects and 
group work as teaching methods because they find that students enjoy those activities more. To 
the extent that projects and group work strategies are not really being guided by pedagogical 
beliefs about learning, but instead by simple classroom survivability, the way that teachers use 
these strategies might differ a great deal and the consequences for student learning may differ as 
well. For example, teachers who use projects and group work but who do not believe in 
constructivist theories of learning may follow a procedurally-scripted approach rather than one in 
which students are being intellectually challenged. Group work assignments may simply involve 



students dividing up their work and sharing answers to worksheet problems. Projects may involve 
little self-direction and require little reflective thought. 

Similarly, teachers who provide students with a cognitively challenging educational experience 
may teach using direct explanations as much or more than other teachers. If so, this would 
suggest that teaching through projects and collaborative group work may not be important 
approaches for providing intellectually enriching school experiences. 

First, we address the prevalence of group work and its relationship to cognitive challenge. Then 
project work is discussed in the same way. 

Group Work - Prevalence and Relation to Cognitive Challenge 

To get a rough estimate of how teachers divide up the time they have for classroom instruction, 
respondents were asked to estimate, out of the most recent five hours of teaching to their sampled 
class, the amount of time spent on each of four activities: teacher-led whole-class discussion, 
student-led discussion or presentations, seatwork assignments, and small group work. Somewhat 
surprisingly, teachers reported more time spent in group work than in any of the other activities. 
Based on teacher self-reports regarding the last five hours taught, only computer and business 
teachers report more seatwork than group work, and only foreign language and math teachers 
report more whole-class teacher-led instruction than group work. In fact, a majority of all 
teachers reported that students worked in groups for more than one hour out of the last five. 
Although in answer to another survey question, more teachers reported that their students did 
seatwork on a daily basis than reported daily ’’work in small groups to come up with a joint 
solution or approach to a problem" (20% vs. 14%), if one were to add in other purposes of small 
group work, the total daily use of group work might easily exceed the number reporting daily 
seatwork. 

Teachers find that older students’ have a greater ability to conduct their activities in groups for 
extended periods of time. Elementary teachers generally report doing less group work than do 
secondary teachers. More than half (56%) of the secondary academic teachers report more than 
an hour of group work in the last five, compared to 41% of elementary teachers. 

Among secondary teachers, more than two-thirds of science teachers have students work in 
groups (or pairs), probably in lab settings. It was business teachers who reported the least time 
having students work in groups (only 29% for more than an hour in the last five, less than any 
other subject). Interestingly, business teachers were also least likely to spend more than an hour 
having students give presentations or lead discussions. In fact, only 1% of business teachers 
report this activity as compared to 21% of vocational teachers. Given the importance of working 
in teams and making presentations in business settings, this finding is striking. 
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TABLE 14: FREQUENCY OF GROUP WORK ACTIVITIES BY SUBJECT AND LEVEL TAUGHT 





Specific Group Work Items (percent meetinq criterion) 


Mean Z-score on 
Group Work Index 


Subject & Level* 


Students worked in 
groups for more than 
one hour out of the 
last five 


Small group 
solutions to 
problems (weekly or 
more) 


Introduced current 
unit with small group 
discussion 


Science (MS) 


70 


48 


44 


0.26 


English (HS) 


51 


37 


49 


0.17 


Elementary 


41 


56 


41 


0.16 


Vocational (All) 


69 


50 


44 


0.15 


Social Studies (HS) 


59 


38 


39 


0.02 


Math (MS) 


57 


59 


29 


0.02 


Math (HS) 


54 


45 


29 


-0.05 


Science (HS) 


66 


37 


31 


-0.09 


English (MS) 


41 


32 


37 


-0.11 


Foreign Language (All) 


37 


39 


34 


-0.19 


Social Studies (MS) 


46 


29 


36 


-0.24 


Computers (All) 


55 


34 


33 


-0.29 


Fine Arts (All) 


62 


21 


20 


-0.55 


Business (All) 


29 


25 


19 


-0.76 


All Teachers 


51 


44 


37 


0.00 



Sample: Probability sample. 

*MS = middle school, HS = high school. 



For some academic subjects, differences between middle school and high school level teachers 
were notable. It seems clear that in science, group work is more prevalent in middle schools while 
in social studies it is more prevalent at the high school level. (See Table 14.) Not only do high 
school social studies teachers do more group work, but they also report less seatwork than middle 
school social studies teachers. (Only 19% reported more than an hour of seatwork compared to 
48% of middle school social studies teachers.) 

In a second survey question about the frequency of group work (mentioned briefly earlier), 
teachers were asked about how often they have students work in groups in order to come up with 
a joint solution or approach to a problem. At the secondary level, 44% of academic teachers 
reported that they had students work in problem-solving teams on a weekly basis. This activity is 
most commonly found among math teachers, particularly middle-school math teachers, 59% of 
which report group problem-solving at least weekly. Science teachers are the next most likely to 
have students work in this type of group, but this also occurs more often in middle school classes. 
Most elementary teachers also reported using this teaching strategy, but, as with other frequency 
measures, it is difficult to compare the answers of the elementary teachers because most of them 
taught self-contained (whole-day) classes and thus had more time to implement a variety of 
teaching methods. (See Appendix Table A-5.) 

The last question about group work in the TLC survey concerns whether, as a means of 
introducing a new unit to students, the teacher divided students into small groups to discuss topics 
related to the new unit. Overall, just over one-third (37%) said they did so on the current lesson 
being taught. Introducing a unit this way, which is probably more time-consuming than other 
approaches, tended to happen more at the middle school level and varied by subject. For example, 
middle school science teachers used group discussions as unit starters more than high school 
science teachers did. However, in contrast, high school English teachers used groups in this way 
more often than middle school English teachers did. These school-level differences within 
subjects parallel what was found for the other group work items in Table 14. 
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Relationship of Group Work to Cognitive Challenge , by Subject and Level Taught 

So far, we have described the extent and variety of group work that occurs in the different 
subjects. However, a more central question for understanding teacher pedagogy is to learn to 
what extent small group work is associated with cognitively challenging tasks rather than being 
reflective of "mere activity." To do this, we compare scores on our Cognitive Challenge Index of 
teachers who make substantial use of group work (i.e., teachers who are above the 50 th percentile 
for their subject-matter in overall group activity) and Cognitive Challenge scores of teachers in 
the same subject who use small group strategies less often. 

The findings in Table 15 clearly support the idea that teachers whose students do group work are 
much more oriented towards providing instruction that is cognitively challenging than are 
teachers who use group work strategies less often, if at all. Across all subjects, the effect size 
between above-average group-work-using teachers and below-average users on Cognitive 
Challenge is +0.9, or close to a full standard deviation. Within each subject and school level 
(every one except secondary foreign language), teachers who use group work more than the 
typical teacher in that subject average at least 0.7 standard deviations higher on Cognitive 
Challenge than teachers who are less active in using student groups. (Analogous correlation 
coefficients to these effect size measures are presented in Appendix Table A-6.) 

Among the three specific indicators of group work, the highest effect size was found for the item 
distinguishing between teachers who had students solve problems in small groups on a weekly 
basis versus those who used that method less often or not at all (E.S. =+0.9). 14 In fact, for a 
majority of subject-level categories, it was this activity of group problem-solving that had the 
highest effect size on Cognitive Challenge. Group problem-solving is particularly associated with 
high Cognitive Challenge among social studies teachers (both middle and high school), middle 
school math teachers, and elementary teachers. The effect sizes for all four of those groups was 
greater than 1.0 standard deviation units. 

The strategy of using group discussions as part of unit introductions is also associated with 
substantially higher Cognitive Challenge. Like the first item discussed, this relationship was true 
for teachers of every subject and level studied except for foreign language teachers. The use of 
group discussions to introduce a unit seems particularly likely to be motivated by a constructivist 
theory of learning, which argues as its central thesis that prior student understandings should be 
the starting point for instruction. This approach was particularly associated with high Cognitive 
Challenge in computer classes (E.S. = 1.1). 

By comparison, reporting an hour or more of group activity in the last five was somewhat weaker 
than the other two items in terms of predicting Cognitive Challenge (E.S. =+0.5). In some 
subjects, (e.g., vocational-education, high school social studies and math, and middle school 
science), teachers may routinely be able to spend an hour or more having students work in 
groups, without engaging groups in reflective writing activity, posing probing questions, or 
having them engage in problem-solving. Even so, for other subject-level combinations — middle 
school math and high school English, in particular — the teachers who were more likely than 
average to assign group work to their selected class over the previous five class meetings were 
also the teachers whose practice involved more cognitively challenging activities. 



14 Although the survey item has explicit "problem-solving” elements in it, the effect size was nearly as 
strong (E.S. =0.7) when only considering the Meaningful Thinking sub-scale — i.e., excluding the problem- 
solving items. 




'/ . 

■W 



33 



37 



TABLE 15: EFFECT SIZE OF GROUP WORK ON COGNITIVE CHALLENGE 



Cognitive Challenge E.S. by Group Work Practices Cognitive Challenge 



(criterion met vs. not met) E.S. by Group Work 



Subject & Level** 


Work in small 
teams >1 hr in 
last 5* 


Small group 
solutions to 
problems weekly 


Introduced new 
unit w/groups 
this time 


Index (Top 50% on 
index vs. bottom 
50%) 


Science (MS) 


0.2 


0.8 


0.7 


0.8 


English (HS) 


0.9 


1.0 


0.3 


0.7 


Elementary 


0.6 


1.1 


0.7 


1.0 


Vocational (All) 


-0.1 


0.8 


0.8 


0.7 


Social Studies (HS) 


0.2 


1.1 


0.7 


0.8 


Math (MS) 


0.8 


1.1 


0.7 


1.1 


Math (HS) 


0.3 


0.7 


0.5 


0.9 


Science (HS) 


0.5 


0.8 


0.8 


1.0 


English (MS) 


0.4 


1.0 


0.8 


0.9 


Foreign Language (All) 


* 


0.2 


0.4 


0.2 


Social Studies (MS) 


0.5 


1.1 


0.8 


0.9 


Computer (All) 


0.4 


0.9 


1.1 


1.0 


Fine Arts (All) 


* 


0.8 


0.9 


0.8 


Business (All) 


* 


* 


* 


1.3 


All Teachers 


0.5 


0.9 


0.7 


0.9 



Sample: Probability and purposive samples. Data in first column are from questionnaire versions 3 and 4 only. 

* Too few cases either meeting frequency of use criterion or not meeting criterion for a valid comparison. (Gen- 
erally, Raw N < 25.) 

**MS = middle school, HS = high school. 



Student Projects and Hands-on Activities 

Student projects constitute a fairly wide range of teacher-directed learning activities. What are 
called "projects" by different teachers vary in their duration; in the number and types of different 
tasks involved; in how much variation between students in project content that a teacher permits; 
and in the degree of independence, creativity, and initiative encouraged. Nevertheless, compared 
to other common types of classroom assignments, projects tend to be much longer in duration, 
involve a wider variety of tasks, permit greater variation among students in the work that they do, 
and provide for greater independence, creativity, and initiative than other school assignments. 

Overall, about one-half of the teachers in our study (48%) report that students in their selected 
class are involved each month in projects of a week or longer in duration. Elementary teachers are 
more likely to report having their students do projects than are secondary teachers of academic 
subjects (56% vs. 39%). That difference is likely to be in large part due to the fact that most 
elementary teachers have the same students in class for many more hours than secondary teachers 
do. The difference in project use between elementary teachers with self-contained classes and 
other elementary teachers is nearly as great (60% vs. 47%) as the difference between all 
elementary teachers and secondary teachers of academic subjects. 

Among secondary teachers, projects are a more central part of teaching in the applied subjects 
than in academic subjects. Projects lasting a week or longer were most common among fine arts 
and computer teachers (92% and 74%, respectively), but also occurred in a majority of vocational 
education and business education classes. However, they were also part of the teaching practice 
of a majority of English teachers, both at the middle school level (60%) and high school level 
(64%; see Table 16). From a different perspective, given the larger numbers of teachers in 
academic subjects, it is still the case that more than half of all student project activity in 
secondary schools takes place in English, science and social studies classes. (See Table A-7.) 
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TABLE 16: FREQUENCY OF PROJECT ACTIVITY BY SUBJECT AND LEVEL TAUGHT 



Subject & Level* 


% Week-long 
Projects 
Monthly + 


% Hands-on or 
Labs Weekly + 


% Creation of 
Product for Use by 
Others Monthly + 


% Demonstrate for Overall Projects 
Outside Audience Z-score 

Monthly + (all 4) 


Fine Arts (All) 


92 


91 


21 


28 


1.33 


Vocational (All) 


66 


83 


53 


30 


1.11 


Computer (All) 


74 


97 


36 


15 


0.97 


Business (All) 


59 


80 


4 


18 


0.30 


Elementary 


56 


61 


12 


12 


0.16 


Science (MS) 


38 


70 


14 


9 


0.09 


English (HS) 


64 


33 


9 


16 


-0.06 


English (MS) 


60 


33 


8 


10 


-0.14 


Science (HS) 


39 


65 


4 


5 


-0.14 


Foreign Language (All) 


27 


29 


29 


11 


-0.21 


Social Studies (MS) 


48 


34 


9 


10 


-0.23 


Social Studies (HS) 


39 


27 


5 


10 


-0.34 


Mathematics (MS) 


13 


24 


9 


6 


-0.68 


Mathematics (HS) 


7 


18 


2 


1 


-0.92 


All Teachers 


48 


51 


12 


11 


0.00 



Sample: Probability sample. 

*MS = middle school, HS = high school. 

Another survey question asked teachers about an instructional practice similar in many ways to 
weeklong projects: the use of "hands-on or laboratory" activities. Like projects, hands-on work is 
meant to give students an opportunity to apply ideas in concrete ways — to make use of 
knowledge rather than treat it as a set of abstract concepts. Like week-long projects, hands-on 
work is very common in applied secondary courses (80 to 97% reporting weekly hands-on 
activity), but are nearly as frequent in science classes (65-70% weekly). However, the concept of 
"hands-on" may not have wide understanding in other academic subjects, since at most one one- 
third of the teachers of other academic subjects (social studies, math, English, and foreign 
languages) reported giving students even monthly opportunities to engage in hands-on work. 

The one notable difference between middle and high school teachers concerns social studies 
teachers. Social studies teachers in middle school spend more class time on student project-type 
activity, both the weeklong projects we asked about in one question and the "hands-on" work 
from the other question. 

In addition to week-long projects and hands-on activity, we also asked about more ambitious, and 
consequently less common, types of projects — ones that involve students making a product for 
use by others and ones in which they demonstrate their work to audiences other than their own 
school or family. Only about 10% of all teachers reported having their selected class do these 
activities to any extent. (See Table 17.) 



TABLE 17: PERCENT OF TEACHERS REPORTING DIFFERENT TYPES OF PROJECT ACTIVITIES 



Project Activities Overall 
Reported... 




How often do students... 




Work on 
projects that 
take a week or 
more 


Do 

hands-on/ 

laboratory 

activities 


Create a 
product for use 
by someone 
else 


Demonstrate their 
work to audience 
beyond school/ 
home 


Never 


10 


8 


46 


53 


Sometimes 


42 


20 


41 


35 


1-3 times per month 


28 


21 


9 


9 


1-3 times per week 


8 


32 


3 


2 


Almost everyday 


11 


19 


0 


0 


Total 


100 


100 


100 


100 



Sample: Probability sample. 
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As with project work in general, it is clearly the case that applied teachers more often have 
students create products for use by others or demonstrate their work to an outside audience. 
Vocational teachers employ these approaches most often, with more than 50% saying their 
students produced a product for use on a monthly basis and 30% saying their students 
demonstrated their work before an outside audience. Next to vocational education teachers, 
products made for use were most often part of computer teachers’ practices (36%, monthly), 
while demonstrations of work to outsiders were most commonly done by students in fine arts 
classes (28%, monthly). In contrast, fewer than 10% of the secondary academic subject-matter 
teachers said their students produced products for use (monthly) and only slightly more said their 
students demonstrated their work before outside audiences. Among the academic teachers, 
demonstrations were most common among high school English teachers, but even there, only one 
in seven reported this as part of their practice (14%). 

In sum, a far greater proportion of teachers in the applied subjects have students doing project 
work than in academic classes. The one exception is in English, where most students are asked to 
work on projects that take a week or longer; and in science, where most teachers have students do 
laboratory and other hands-on work on a weekly basis. Overall, more project-based activities 
occur in middle school academic classrooms than in high school academic classrooms. We 
suspect that high school academic teachers may have more curriculum pressures and lack the 
curricular flexibility of middle school teachers. 

Project Activity and Cognitive Challenge: 

All four aspects of project work measured are moderately to highly associated with higher levels 
of cognitive challenge. Overall, the largest effect size is for students creating products for use by 
others (E.S.=0.7). 15 However, the other three items' effect sizes were all between 0.5 and 0.6. 

The relationship between project use and cognitive challenge varies a great deal by subject. 
Business education teachers who are above the median in project use have only slightly higher 
scores on cognitive challenge than other business education teachers, suggesting that projects are 
not particularly a vehicle for raising the intellectual level of subject-matter content in business 
courses. On the other hand, among English, science, middle school math, and elementary 
teachers, those who do more projects are close to if not a full standard deviation higher on use of 
cognitively challenging strategies than teachers in the same subjects who are less active in the use 
of student projects. 16 In these subjects, projects indeed may be an important mechanism whereby 
cognitively-oriented teachers enable students to have a more meaningful learning experience. 

The cognitive challenge of particular types of project activities (e.g., demonstrations, making 
useful products, etc.) appears to vary subject-by-subject. Foreign language teachers' use of 
language labs and high school math teachers’ use of hands-on activities seems not to be related to 
cognitive challenge whereas for other academic subjects and even for middle school math, the 
teachers who give students projects and hands-on activities to work on do report greater use of 
cognitively challenging teaching strategies. (See Table 18.) 



15 This analysis contrasts teachers who report that product-for-use and demonstration-to-outsider activities 
were part of their teaching practice at all versus teachers who did not do these things. The earlier 
descriptive analysis dichotomized these variables at "monthly or more" versus less often or not at all. 
However, because even monthly frequency is so rare, dichotomizing at "monthly or more" would have 
produced too few cases for most subjects to permit calculation of reliable effect sizes. 

16 Correlation coefficients in Appendix Table A-6 produce essentially the same results. 
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TABLE 18: EFFECT SIZE OF PROJECT ACTIVITY ON COGNITIVE CHALLENGE 



Cognitive Challenge E.S. by Project Practice 
(criterion met, vs. not met) Cognitive 



Subject & Level** 


Week long 
projects 
monthly + 


Hands-on/Labs 
weekly + 


Make product 
for use by 
others 

sometimes + 


Demonstrate 
work to an 
audience 
sometimes + 


Challenge E.S. by 
Project Index 
(top 50% on index 
vs. bottom 50%) 


Fine Arts (All) 


* 


* 


1.3 


♦ 


0.8 


Vocational (All) 


0.9 


* 


* 


0.9 


0.8 


Computer (All) 


0.4 


* 


0.7 


0.9 


0.6 


Business (All) 


0.2 


* 


0.4 


0.6 


0.1 


Elementary 


0.9 


0.7 


0.7 


0.6 


0.9 


Science (MS) 


0.6 


0.8 


0.8 


0.4 


0.8 


English (HS) 


0.6 


0.6 


0.9 


0.9 


0.9 


English (MS) 


0.7 


0.7 


0.7 


0.7 


0.9 


Science (HS) 


0.8 


0.6 


0.5 


0.7 


0.9 


Foreign Language (All) 


0.5 


-0.1 


0.3 


0.8 


0.4 


Social Studies (MS) 


0.6 


0.7 


0.8 


0.6 


0.7 


Social Studies (HS) 


0.7 


0.8 


0.5 


0.3 


0.6 


Math (MS) 


0.4 


0.7 


0.8 


0.8 


0.8 


Math (HS) 


0.6 


0.1 


0.6 


0.5 


0.5 


Total 


0.6 


0.5 


0.7 


0.6 


0.8 



Sample: Probability and purposive samples. 

* Too few cases either meeting frequency of use criterion or not meeting criterion for a valid comparison. (Generally, 
Raw N < 25.) 

** MS = middle school, HS = high school. 



Comparing and Combining Projects and Group Work: 

We examined the co-incidence of projects, group work, and activities involving cognitive 
challenge and found that, indeed, within every subject, the teachers who are most actively 
engaged in both project and group work in their classes provide substantially more activities 
associated with cognitive challenge than other teachers do. For example, compared to the average 
high school English teacher, teachers who are above-average in both the use of projects and group 
work have average z-scores that put them into the 82 nd percentile (among high school English 
teachers) on assigning cognitive challenging tasks to their students. Similar results were found 
for every academic subject and for computer education teachers as well. In contrast, teachers 
who are below-average on use of both projects and group work also are far below average on 
assigning cognitively challenging tasks. (See Table 19.) 

The effect sizes of both group work and projects on cognitive challenge are substantial (0.9 and 
0.8, respectively). For most subjects, projects and group work are about equally associated with 
cognitive challenge. However, for mathematics, computer classes, and high school social studies, 
group work is more associated with high cognitive challenge than project work is. In those 
subjects, projects may be fairly superficial while group work may form the basis of serious 
discourse about understanding the content of the subject. 17 

Overall, it is the presence of above-average frequency of both projects and group work that most 
strongly predicts whether a teacher provides high cognitive challenge, according to our measure. 



17 One distinction comes from the exploratory factor analyses. While project activities consistently loaded 
on a separate factor, group work items often loaded with other items linked with cognitive challenge. 
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Generally, the effects of the two components are additive — a teacher who is above-average on 
either projects or group work employs cognitively challenging tasks more than one who is below- 
average. The large correlations among these three components of constructivist pedagogy, among 
teachers of the same subject area, suggests that teachers who view learning as a meaning- 
constructing process will generally be the same teachers who prioritize conceptual understanding 
over factual learning and routine skill competencies and will tend to incorporate all three 
pedagogical practices into their teaching — student projects, students working in groups, and tasks 
that provide cognitive challenge. 



TABLE 1 9: WITH IN-SUBJECT COGNITIVE CHALLENGE SCORES, BY TEACHERS SCORING 
ABOVE OR BELOW THE MEAN OF SAME-SUBJECT TEACHERS ON PROJECTS AND GROUP WORK 



Above mean in either projects or group work 



Subject & Level* 


Neither 


Projects only 


Group work only 


Both 


Total 


English (M) 


-.76 


-.12 


-.10 


.62 


-.09 


English (H) 


-.66 


.13 


.03 


.90 


.10 


Social Studies (M) 


-.51 


-.35 


-.22 


.54 


-.09 


Social Studies (H) 


-.51 


-.02 


.27 


.62 


.12 


Science (M) 


-.44 


.18 


.13 


.84 


.26 


Science (H) 


-.68 


-.19 


-.07 


.73 


-.17 


Math (M) 


-.61 


-.03 


.30 


.94 


.20 


Math (H) 


-.68 


-.22 


.16 


.53 


-.20 


Computer (All) 


-.68 


-.20 


.18 


.81 


.00 


Elementary 


-.75 


-.02 


.01 


.72 


.00 


All Teachers 


-.64 


-.07 


.08 


.71 


.00 



Sample: Probability and purposive samples. 

*MS = middle school, HS = high school. 

TRADITIONAL INSTRUCTION: PREVALENCE AND COGNITIVE CHALLENGE 

The final set of survey items dealing with teacher pedagogy that we examined are a set of 
measures of fairly common teaching practices that are generally associated with a skills-practice 
and knowledge-transmission orientation rather than with constructivist pedagogical beliefs and 
practices. These include frequency of teacher-led whole-class discussions, individual seatwork 
("answering questions in the textbook or worksheets"), asking questions in order to test students' 
knowledge of the correct answer, and having students do introductory drills to introduce them to a 
new unit. 

Do these traditional activities represent an independent aspect of a teacher’s overall repertoire, 
one that most teachers can be expected to follow regardless of how much they use projects and 
group work or whether or not they emphasize cognitively challenging strategies? Or do they 
represent an alternative view of instruction that occurs mainly among teachers who do not employ 
the constructivist-compatible practices that we have already discussed? 

We briefly present the prevalence of the specific skills- and transmission-oriented practices in our 
survey, and then show the relationship between these practices and the use of projects, group 
work, and cognitive challenge, both between subjects and within subjects. 

Of the transmission-oriented activities we asked about, one very common practice involves the 
use of textbooks and worksheets. Two-thirds of all teachers report assigning worksheets or 
textbook exercises to students in their selected class. Nearly as many (64%) often introduce a 
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new unit to students by having them do introductory drills on related skills or facts. A majority of 
teachers "very often" or "always" ask students questions in order to check that students know the 
"correct" answer. And most teachers spend some time in whole-class discussion mode, asking 
students a series of inter-related questions, though only one third did so for more than one hour in 
the last five. Table 20 shows the percentage of teachers by subject and level who reported using 
these practices frequently. However, the z-score index in the last column of the table is based on 
how infrequently these practices are used; thus it represents "infrequent use of traditional 
practices." 



TABLE 20: USE OF TRADITIONAL PRACTICES BY SUBJECT AND LEVEL TAUGHT (PERCENT) 



More than one 
hour of teacher-led 
recitation in the 
Subject & Level* past five hours 


Ask questions to 
check for correct 
answer very often 
or more 


Often use drills 
to introduce a 
unit or did so for 
the current unit \ 


Have students 
answer questions 
from text or 
worksheet monthly + 


Mean Z- 
score on 
Index 
(reversed) 


English (HS) 


25 


30 


49 


49 


0.68 


Fine Arts (All) 


16 


48 


77 


21 


0.66 


Vocational (All) 


13 


43 


62 


62 


0.32 


Computer (All) 


4 


54 


87 


39 


0.31 


Science (MS) 


31 


48 


61 


65 


0.24 


English (MS) 


23 


52 


59 


66 


0.19 


Social Studies (MS) 


43 


53 


62 


57 


0.11 


Science (HS) 


43 


48 


60 


72 


0.09 


Social Studies (HS) 


41 


52 


64 


62 


0.06 


Elementary 


29 


54 


59 


79 


-0.11 


Business (All) 


9 


71 


87 


67 


-0.26 


Math (MS) 


64 


68 


76 


84 


-0.55 


Foreign Language (All) 


43 


77 


86 


80 


-0.58 


Math (HS) 


69 


64 


79 


88 


-0.70 


All Teachers 


33 


54 


64 


68 


0.00 



Sample: Probability sample. 

**MS = middle school, HS = high school. 

The classes where these transmission-oriented practices are least frequent are high school 
English, computer classes, vocational education, fine arts, and middle school science. 
Interestingly, all five of those classes have the highest mean scores on one of the components of 
constructivist practice already discussed: High school English classes are highest on cognitive 
challenge; fine arts, vocational education, and computer classes have the three highest mean 
scores on projects; and middle school science teachers use group work more than any other 
category of teacher. In contrast, the subjects where knowledge-transmission- and skills-oriented 
practices are most common are the same subjects where the previously identified components of 
constructivist practice are least common — mathematics (lowest in the use of projects), business 
education (lowest in both cognitive challenge and group work), and foreign language (among the 
lowest in cognitive challenge and low on projects as well). (See Table 21, which shows the rank- 
order of each of 15 groups of teachers on all of the measures of constructivist teaching practice, 
including "infrequent use of traditional practices" which we are discussing here.) 18 

These findings strongly suggest that how frequently teachers of a given subject use skill s-practice 
and transmission activities is inversely related to how frequently teachers of that subject follow a 
constructivist teaching practice. 19 



18 For the actual mean z-scores on each of the constructivist practice components for each subject and level 
combination shown in Table 21, see Table A-8. 

19 We do not want to deny the value of traditional methods for producing learning under the right 
circumstances, for example when time is short and specific skills are desired. The question is more the 
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TABLE 21: RANKING OF SUBJECTS/LEVELS 
BY EACH COMPONENT OF CONSTRUCTIVIST TEACHING PRACTICE 



Constructivist Practice 





Cognitive Challenge 




Active Learning 




(Constr- 


Subject & Level* 


Meaningful 

Thinking 


Problem- 

Solving 


(Cognitive 

Challenge 

Index) 


Group 

Work 


Projects 


Infrequent 

traditional 

practices 


(Active 

Learning 

Index) 


uctivist 

Practice 

Index) 


English (MS) 


2 


11 


3 


10 


9 


6 


8 


4 


English (HS) 


1 


8 


1 


3 


8 


1 


4 


1 


Social Studies (MS) 


7 


13 


10 


12 


12 


7 


12 


11 


Social Studies (HS) 


4 


10 


5 


6 


13 


9 


10 


8 


Science (MS) 


6 


4 


6 


1 


6 


5 


5 


5 


Science (HS) 


9 


9 


9 


9 


10 


8 


9 


10 


Math (MS) 


11 


1 


8 


7 


14 


13 


14 


12 


Math (HS) 


14 


6 


13 


8 


15 


15 


15 


15 


Foreign Lang. (All) 


10 


14 


12 


11 


11 


14 


13 


13 


Computer (All) 


12 


7 


11 


13 


3 


4 


3 


7 


Business (All) 


15 


15 


15 


15 


4 


12 


11 


14 


Vocational (All) 


8 


3 


7 


4 


2 


3 


2 


3 


Fine Arts (All) 


13 


12 


14 


14 


1 


.2 


1 


9 


Elementary Self- 
contained 


3 


2 


2 


2 


5 


11 


6 


2 


Elementary Other 


5 


5 


4 


5 


7 


10 


7 


6 



Sample: Probability sample. 

*MS =middle school, HS = high school. 



The previous conclusion concerned comparisons of teachers in different subjects — i.e., subjects 
whose teachers reported infrequent use of traditional practices were the same subjects where 
teachers reported frequent use of cognitively challenging practices. However, the question 
remains about the pattern within each subject: In any given subject, are the teachers who most 
infrequently use transmission and skills-practice activities the same teachers who most frequently 
employ practices that emphasize cognitive challenge? (And vice versa — do the heavily traditional 
teachers score lowest on Cognitive Challenge ?) 20 



quality of the learning that is taking place, and the meaningfulness of the goals. When they are developed 

and used effectively, there are instructional methods that are consistent with so-called “traditional teaching” 

that could engender deep thinking in students under specific circumstances, for example an inspirational 

lecture before an activity where important issues are raised. Are we saying a teacher should never lead the 

whole class in a discussion? No, just that the amount of time spent on that does take away from other 

activities. Judicious use of traditional methods may be entirely appropriate at times. Clearly, this survey 

emphasized constructivist-compatible practices and did not attempt to measure the quality of so-called 

traditional instruction as carefully as it did the many faces of constructivist teaching. It is possible that if 

we had asked enough questions, we would have found traditional-compatible practices that could promote 

deep thinking, or that would impact some other measures. 

20 • * 

The same question can be asked about the relationship between infrequent use of traditional practices and 
frequency of use of projects and of group work. The correlations among the three components of what we 
call "Active Learning Strategies" are shown in Appendix A-4, separately for each subject-level 
combination. Correlations between infrequent traditional practices and use of projects are quite high, 
averaging nearly 0.4 for the 15 categories of teachers examined, nearly the identical average as found for 
the correlation between projects and group work. However, the correlations between infrequent use of 
traditional practices and group work are substantially smaller, averaging only 0.13 for the same categories 
of teachers. 
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The answer is that academic teachers who reported the least frequent use of traditional 
transmission and skills-practice activities scored somewhat higher on the Cognitive Challenge 
Index (and vice versa). However, the pattern is not uniform across all academic subjects nor is it 
true for the applied secondary subjects. The strongest patterns (with effect sizes of 0.5 or higher) 
are for foreign language teachers, middle school math and English teachers, and high school 
English teachers. In those subjects, the teachers who frequently use transmission and skills- 
practice activities are the least likely to have high Cognitive Challenge scores. However, in other 
academic subjects — science, high school social studies, and high school math — the relationships, 
although positive, are smaller. 21 In other words, it may be more possible for direct instruction 
approaches and cognitive challenge to co-exist in those particular subjects and levels. And in the 
applied subjects, there is either no relationship (as among fine arts, vocational education, and 
computer teachers) or even the opposite one (among business teachers). That is, for business 
education teachers, those who most frequently report using traditional skills-practice activities 
also are the ones who most frequently report using cognitively challenging ones (see Table 22). 22 



21 In this case, a few of the correlation coefficients produce somewhat different results. In particular, the 
correlation for middle school math teachers is not as high as the effect size would indicate (r=0.2). See 
Appendix Table A-6. 

22 The smaller association between infrequent use of transmission practices and frequent use of cognitively 
challenging ones (compared to the associations reported among other pairs of components) is at least partly 
due to the effects of questionnaire response style on measured effect sizes. Here we are finding small 
positive asssociations between low frequency of one type of practice and high frequency of another type of 
practice, whereas the other findings (high positive associations) are between high frequency of two types of 
practices. Thus, whereas in the other findings response style patterns resulted in measured associations that 
are probably higher than are true, in fact, in this case the response style effects work to diminish the size of 
the association. Consequently, to the extent that response style patterns are operating, the true effect sizes 
between infrequent traditional practices and Cognitive Challenge are larger (more positive) than the 
measured effect sizes. 
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TABLE 22: EFFECT SIZE OF THE AVOIDANCE (INFREQUENCY) OF TRADITIONAL PRACTICES 

ON COGNITIVE CHALLENGE 

Cognitive Challenge E.S. by Infrequency of Traditional Practices 
(criterion met vs. not met) Cognitive 



Subject & Level** 


One hour or 
less of whole 
class 

discussion 


Not testing 
student 
knowledge of 
"correct answers" 
very often 


Did not or 
rarely use 
introductory 
drills 


Individual work in 
textbooks or 
worksheets less 
than monthly 


Challenge E.S. by 
Traditional 
Practices Index 
(bottom 50% on 
index vs. top 50%) 


English (HS) 


0.5 


0.2 


0.6 


0.4 


0.5 


Fine Arts (All) 


* 


0.2 


0.1 


-0.1 


0.1 


Vocational (All) 


* 


0.0 


0.2 


-0.1 


0.1 


Computer (All) 


* 


0.2 


-0.4 


-0.2 


-0.2 


Science (MS) 


* 


0.1 


0.0 


0.4 


0.3 


English (MS) 


0.2 


0.6 


0.3 


0.6 


0.6 


Social Studies (MS) 


0.2 


0.3 


0.0 


0.4 


0.4 


Science (HS) 


0.6 


0.2 


-0.1 


0.2 


0.2 


Social Studies (HS) 


0.2 


0.1 


0.5 


0.2 


0.2 


Elementary 


0.3 


0.4 


0.3 


0.5 


0.4 


Business (All) 


* 


-0.3 


* 


-0.7 


-0.5 


Math (MS) 


0.2 


0.3 


0.1 


0.0 


0.6 


Foreign Language (All) 


* 


0.5 


* 


0.3 


0.7 


Math (HS) 


0.3 


0.0 


-0.3 


0.1 


-0.1 


All Teachers 


0.3 


0.2 


0.1 


0.2 


0.3 



Sample: Probability and purposive samples. Data in first column are from questionnaire versions 3 and 4 only. 

Too few cases either meeting frequency of use criterion or not meeting criterion for a valid comparison. (Generally Raw 
N < 25.) 

**MS = middle school, HS = high school. 




' £2 



46 



PART III. THE RELATIONSHIP BETWEEN TEACHER BELIEFS AND PRACTICES 



In this report, we have presented information about how a national sample of teachers across 
every major subject and school level (in grades 4-12) responded to survey questions regarding 
their beliefs about good teaching and their instructional practices in one particular class. For the 
most part, we presented descriptive findings about individual survey items, broken down by 
subject and level, although in addition, items were aggregated into seven indices based on a series 
of exploratory factor analyses. 23 An argument was presented suggesting that these indices 
represent distinct, yet correlated, aspects of an underlying global dimension of pedagogy that 
ranges from a primary reliance on and belief in an information transmission and skills-practice 
model of pedagogy to a belief in and use of instructional practices more compatible with a 
constructivist theory of learning. We presented evidence that a strong relationship exists between 
the component of instructional practice labeled Cognitive Challenge and other aspects of a 
constructivist teaching practice — use of projects, group work, and (to a lesser extent) a relatively 
infrequent reliance on traditional transmission and skills-practice strategies. To complete our 
understanding of this aspect of teacher pedagogy, we need to understand the relationship between 
teachers' beliefs about good teaching and their actual use of various practices in the classroom. 

How teachers organize classroom activities is a function of many influences — external pressures 
to cover curriculum or to prepare students for examinations, the expectations of a teacher's 
colleagues and immediate supervisors, the skills and confidence a teacher has that different 
instructional approaches can be managed in a diverse classroom with space, resource, and 
scheduling limitations, and many other factors. However, teachers are also likely to feel 
accomplished when they can implement classroom practices that are consistent with what they 
regard as good and important teaching — when they can teach students about content that they 
themselves believe is important for students to know and when they can provide a collective 
learning experience that they think is likely to result in students growing and maturing in various 
ways, both intellectually and socially. 

To what extent do teachers’ actual classroom practices reflect their strongest beliefs about good 
teaching? Are teachers’ practices constrained greatly by external pressures or by a "realistic" 
understanding about what kinds of instruction can successfully be employed in the particular 
classroom settings in which they teach? By asking teachers to answer questions both about their 
teaching philosophy and about how they teach in the specific class "where you are most satisfied 
with your teaching — where you accomplish your teaching goals most often," we hoped to get a 
rough handle on the extent of this relationship. 

There are two questions involved. One is the absolute level to which a teacher implements her 
philosophy in practice. The other is the extent to which philosophy and practice are correlated 
across teachers — that is, "How much more likely are teachers with a more constructivist 
philosophy to teach in a more constructivist way?" Both of these questions are addressed here. 

One approach to looking at these issues might be to compare individual survey questions about 
beliefs with "parallel" questions dealing with classroom practice. Although the belief and practice 



23 As indicated earlier, factor analyses were used to decide which items to group together, but indices 
combining items were constructed by taking mean values of items, after standardizing each item to have 
similar variance. 
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questions were not designed with parallelism in mind, there are some pairs of items that deal with 
similar issues. 



One "beliefs" question, for example, asked teachers whether they agreed that instruction should 
be built "around problems with clear, correct answers, and around ideas that most students can 
grasp quickly." The "practice" question related to this asked teachers how often their objective for 
asking questions of students was to find out if students knew the correct answer — a practice that 
a teacher agreeing with the belief statement would presumably employ. (See Table 23.) 



TABLE 23: FREQUENCY WITH WHICH TEACHERS ASK STUDENTS QUESTIONS 
IN ORDER TO SEE IF THEY KNOW THE CORRECT ANSWER 



Belief: "Instruction should be built around 


Percent asking questions to try to see if 
students know the correct answer... 




problems with clear, correct answers, and 


Never or 




Very Often or 




around ideas that students can grasp quickly." 


Sometimes 


Often 


Always 


Total 


Strongly agree 


7 


19 


74 


100 


Moderately agree 


8 


22 


71 


100 


Slightly agree 


14 


30 


56 


100 


Slightly disagree 


14 


33 


53 


100 


Moderately disagree 


23 


30 


47 


100 


Strongly disagree 


34 


31 


36 


100 


All Teachers 


18 


29 


53 


100 



Sample: Probability and purposive samples; questionnaire versions 1 , 2, and 4. 



Though virtually all teachers, beliefs aside, report asking questions to see if students know the 
correct answer, those who tend to disagree with such an approach are less likely to do so. In fact, 
only 36% of those who strongly disagree with the approach say they are testing students' 
knowledge "very often" or "always" compared to 74% of those who believe in such a basis for 
instruction (those who strongly agree with the traditional statement). 

We examined another "belief" question that asked teachers to position their own philosophy 
between two competing approaches to instruction; one that poses the teacher as a facilitator of 
student learning who provides opportunities and resources for students to discover or construct 
knowledge for themselves; and the other that describes the teacher's role as one who explains 
knowledge in a structured manner. This belief item was matched to a practice item that asked 
teachers how much time they had spent in the last five hours leading a whole-class discussion 
where students listened and answered questions. Presumably, those who believe in a regimented 
approach to teaching would allocate more class time to teacher-led discussion. In fact, we found 
that nearly one quarter of teachers whose philosophy is closer to that of an "explainer" reported 
more than two hours of teacher-led whole-class discussion in the last five hours while only 6% of 
those favoring the role of "facilitator" reported that teacher-led discussion had occurred for such a 
long period of time. (See Table 24.) 
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TABLE 24: AMOUNT OF TIME SPENT ON TEACHER-LED WHOLE-CLASS DISCUSSION 
AMONG TEACHERS WITH VARYING BELIEFS ABOUT INSTRUCTION 



Belief: 


Percent of teachers leading a whole-class discussion for. . . 


Total 


Under 30 
minutes 


30-60 minutes 


1-1/4 to 2 
hours 


Over 2 hours 


Favors Facilitator 


43 


37 


13 


6 


100 


Neutral 


28 


40 


23 


8 


100 


Favors Explainer 


23 


25 


29 


24 


100 


All Teachers 


32 


34 


21 


13 


100 



Sample: Probability and purposive samples; questionnaire versions 3 and 4. 

We examined yet another pair of items, this time addressing student involvement in planning 
classroom activities. The "belief item asked teachers whether they agreed or disagreed with the 
statement that "it is better when the teacher — not the students — decides what activities are to be 
done." We paired this belief item with one that asked teachers to report how often their students 
actually suggested or helped to plan classroom activities. As expected, teachers’ beliefs manifest 
themselves in actual practice. 



TABLE 25: FREQUENCY WITH WHICH TEACHERS HAVE STUDENTS SUGGEST OR HELP PLAN 
CLASSROOM ACTIVITY BY WHETHER OR NOT THEY AGREE WITH THE PRACTICE 



Belief: "It is better when the teacher 


Percent who have students help plan classroom activities... 




— not the students — decide what 
activities are to be done." 


Never or sometimes 


Monthly + 


Total 


Agree 


84 


16 


100 


Disagree 


66 


34 


100 


All Teachers 


76 


25 


100 



Sample: Probability and purposive samples; questionnaire versions 1, 3 and 4. 



Table 25 shows that more than one third of those who believe that it is a good idea to consider 
student input (those who tend to disagree with the traditional statement) actually have students 
help plan classroom activities at least monthly. This makes them twice as likely to involve 
students as those who don’t believe in the practice (34% vs. 16%). Thus we can show that 
teachers holding certain constructivist philosophies carry that philosophy into their classroom 
teaching. 

In our second analysis of this issue, a set of 10 different constructivist-related instructional 
practices were compared with three teacher belief items that clearly contrast between a 
constructivist and a transmission pedagogy: 

• Whether students learned more knowledge from the inquiry-oriented approach to handling 
class discussion attributed to "Mr. Jones" or from "Ms. Hill’s" direct-instruction approach 
(see above, Figure 1). 

• Whether the teacher believed more in being a facilitator for student discovery or in direct 
instruction ("my job is to explain, to show students how to do the work, and to assign specific 
practice"; see Figure 2). 

• Whether the teacher believed it was better to "have all sorts of activities going on in the 
classroom" (at the same time) or whether classes were better organized around whole-class 
assignments with clear directions and accomplishable over short time intervals (also in Figure 
2). 

Teachers taking opposite positions on these issues were compared in terms of the proportion of 
them who reported having their selected class engage in each of 10 different activities on at least 
a monthly basis — activities such as engaging in week-long projects, keeping journals, suggesting 
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or planning classroom activities, and designing their own problems to solve. Teachers favoring an 
inquiry approach, being a facilitator, and organizing class time around multiple simultaneous 
small-group activities were more likely to report doing week-long projects, student journals, 
designing assignments where students had to "represent the same idea in more than one way," 
hands-on activities, reflective student essays, and, in fact, nearly every one of the practice items 
shown in the table. Overall, those who selected the constructivist belief alternative on each survey 
question were about one-half a standard deviation higher on an index measuring the sum of the 
number of constructivist activities engaged in (monthly) than teachers who chose the traditional 
transmission belief statement (or who selected the middle response). (See Table 26.) 
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Indeed, when we examine the full 13-item Belief Index (see above, page 13), teachers who 
reported using five or more of the practices in Table 26 were a full standard-deviation higher on 
the Belief Index (mean = +0.48) than teachers who reported using at most one of the practices 
(mean = -0.52). 

The analysis of the relationship between teacher beliefs and practices has so far collapsed all 
teachers, regardless of subject or level. However, we have already seen that subject and level 
play a large role in the relevance of certain instructional strategies to a teacher's practice (and, 
presumably, in the relevance of related beliefs). For our final foray into this subject, we examined 
the correlation coefficients between the Belief Index and the various components of constructivist 
practice, separately for 15 groups of teachers defined by subject and school level. 

The correlational patterns show, for most of the 15 subject/level groups, a substantial relationship 
between teacher beliefs and practices at the level of generality of these indices. As shown in 
Table 27, the Belief Index correlates at or above r=0.40 with Cognitive Challenge for English, 
social studies, math, and fine arts teachers at both middle and high school levels, for elementary 
teachers of self-contained classes, and for middle school science teachers. (High school science 
teachers fall just below that criterion, at r=0.37.) The correlations between contructivist beliefs 
and group work are slightly higher (a majority of correlations are at or above r=0.44). The project 
work correlations are lower than the other two (the median correlation is r=0.32); however, the 
projects index is correlated above 0.40 with Beliefs for math teachers, elementary teachers, and 
middle school social studies teachers. Even correlations with "infrequent traditional practices" 
(i.e., the reverse-scored index of transmission- and skill-oriented practices) tend to be in the +0.30 
to +0.50 range (with a median correlation of r=0.38 for the 15 teacher groups). Combining 
constructivist practice indices together produces slightly larger correlations. The median 
correlation between "Active Learning" (combining projects, group work, and absence of 
traditional practices) is 0.48, and when that is combined with Cognitive Challenge for a 
Combined Constructivist Pedagogy Index, the median rises to 0.49. 
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TABLE 27: CORRELATION OF CONSTRUCTIVIST BELIEF INDEX 
WITH COMPONENTS OF CONSTRUCTIVST PRACTICE 



Active Learning Strategies (sub-components) 





Cognitive 

Challenge 


Projects 


Group Work 


Infrequent 

Traditional 

Practices 


Active 

Learning 

(total) 


Combined 
Constructivist 
Pedagogy Index 


English (MS) 


.43 


.29 


.42 


.49 


.43 


.47 


English (HS) 


.48 


.38 


.51 


.49 


.57 


.58 


Social Studies (MS) 


.48 


.62 


.59 


.54 


.72 


.65 


Social Studies (HS) 


.44 


.27 


.40 


.43 


.48 


.51 


Science (MS) 


.40 


.32 


.44 


.46 


.51 


.49 


Science (HS) 


.37 


.41 


.41 


.22 


.48 


.46 


Math (MS) 


.40 


.47 


.47 


.37 


.56 


.51 


Math (HS) 


.45 


.46 


.44 


.34 


.58 


.58 


Foreign Language (All) 


.12 


.24 


.44 


.38 


.48 


.31 


Computer (All) 


.27 


.24 


.41 


.24 


.42 


.37 


Business (All) 


.34 


.30 


.18 


.15 


.36 


.41 


Vocational (All) 


.14 


.32 


.53 


-.20 


.43 


.29 


Fine Arts (All) 


.48 


.14 


.11 


.28 


.23 


.45 


Elementary Self-contained 


.51 


.46 


.44 


.48 


.58 


.58 


Elementary Other 


.34 


.47 


.50 


.41 


.57 


.49 


Median correlation in 
column 


.40 


.32 


.44 


.38 


.48 


.49 



Sample: Probability and purposive samples. 
MS = middle school, HS = high school. 



For some applied secondary school subjects, the Belief Index does not correlate well with some 
of the constructivist teaching practice indices. Among vocational education teachers, 
constructivist beliefs are not correlated with cognitive challenge and they tend to be correlated 
with frequent rather than infrequent use of transmission and skill s-oriented practices. At the same 
time, constructivist beliefs correlate well with group work (r=0.53) and fairly well with projects 
(r=0.32). For business education teachers, constructivist beliefs correlate with the overall 
Combined Constructivist Pedagogy Index very well (r=0.41); however, the two coefficients 
between beliefs and (a) group work and (b) traditional practices are both low. Similarly, for fine 
arts teachers, constructivist beliefs correlate highly with the overall index (r=0.45), but they have 
weak correlations with both projects and group work. 

However, among teachers in every academic subject, the patterns are very consistent: Teachers in 
that subject who hold more constructivist beliefs than others of the same subject and school level 
report substantially higher levels of use of all types of constructivist teaching strategies — from 
cognitively challenging ones to projects and group work and they also report less frequent uses of 
traditional instructional practices. For academic subject teachers, the correlations between the 
Belief Index and the Combined Constructivist Pedagogy Index range as high as 0.65 for middle 
school social studies teachers and go no lower than 0.31 (for foreign language teachers). 

These data show that teacher beliefs do a reasonably good job of predicting patterns of practice. 
Although it would be a mistake to assume that teachers who do not employ a give practice fail to 
employ it because they lack a belief in its value, the relationships shown do suggest that teachers 
are more likely to engage in constructivist practices if they state a belief in their validity. In other 
words, beliefs do help determine practice, and therefore to increase the frequency of 
constructivist practice, one might need to directly address teachers' philosophies. On the other 
hand, if one can encourage and guide teachers to carry out a specific practice (even in the absence 



O 

ERIC 



49 



54 



of supporting belief), this may change their beliefs about what is desirable and what is feasible in 
their teaching practice, thus helping to make other constructivist practices more likely. A future 
report will address changes teachers reported having made over the last three years in their 
teaching practices and the circumstances under which increased constructivist practice is more 
likely to occur. 



OTHER ANALYSES OF TEACHER PEDAGOGY USING THE TLC DATA 

The largely descriptive findings presented in this report is one element in our use of the TLC 
survey data to analyze teacher pedagogy and its relationship to other aspects of teaching. 
Analyses are being conducted in two directions: (1) examination of possible determinants of 
differences in teachers’ pedagogy; and (2) examination of possible outcomes of a constructivist 
teacher pedagogy. 



Determinants of Teacher Pedagogy 

In terms of possible determinants, we are looking at three types of variables that may influence 
pedagogy: 

• teacher background and role orientation (private practice versus collaborative professional 
practice), 

• student ability and socio-economic background, and 

• school professional culture, including aspects of staff development, cohesion, and school- 
level leadership. 

Teacher Background and Role Orientation 

Pedagogical differences by teacher gender, years of teaching experience, and educational 
background have important implications for educational policy. The differences in pedagogy are 
particularly strong by gender and by educational background, and these will be examined in a 
future report. The relationship between having a ’’private-practice” orientation towards teaching 
and a transmission-oriented pedagogy (or between having a "professional" orientation and a 
constructivist pedagogy) has been explored in two TLC Special Reports (Becker and Riel, 1999; 
Riel and Becker, 2000). In particular, a private-practice orientation is likely to make a teacher less 
responsive to influences of and changes within the larger teaching culture, which despite external 
pressures towards accountability, is strongly oriented towards a constructivist practice. Teacher 
leaders who are involved with their peers at school and throughout the profession are among the 
most constructivist teachers in the country. 

Student Background 

It seems reasonable to believe that teachers feel that students of apparent higher ability or greater 
prior knowledge respond more favorably to one type of teaching approach as opposed to another, 
or that teaching in low socio-economic-status communities brings greater pressures for teachers 
to adopt transmission-oriented practices. A future report will examine equity issues in technology 
use and pedagogy. 

School Culture 

Teacher pedagogy may be affected by the cultural environment in which they work — for 
example, staff development opportunities, cohesion and goal uniformity among the teaching staff 
and between administrators and teachers, the extent of external pressure to teach at variance with 
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one’s beliefs, the attitudes of school administrative leaders, and the pedagogy of teachers with 
whom they work. The relationship of many of these variables to teacher pedagogy have already 
been examined in the Becker and Riel (1999) paper. Future work will incorporate more of these 
in greater depth. 

Outcomes of Teacher Pedagogy 

The principal focus of the Teaching, Learning, and Computing study is on the determinants of 
teachers' use of information and communications technologies. In Report 1 of this series (Becker, 
1999) we already showed that teacher pedagogy is a principal variable for identifying which 
teachers use the Internet, both professionally and in directing students in its use. It is not 
surprising, then, that we are finding the same is true of the relationship of pedagogy to computer 
use as a whole: Constructivist-oriented teachers use computers professionally in more varied 
ways, have greater technical expertise in the use of computers, use computers frequently with 
students, and use them in apparently more powerful ways. These findings have been explored in 
several papers to date (Ravitz, Wong, and Becker, 1999; Riel and Becker, 2000; and Becker, 
2000a). Further examination of the relationship between teacher pedagogy and computer use will 
be the topic of a future report. 

Finally, although we have already identified teacher role-orientation as a likely determinant of 
teacher pedagogy, it seems plausible to believe that a teacher's approach to her practice in turn 
also affects her role-orientation. Thus, while having a "private-practice" role orientation is likely 
to engender a transmission-oriented practice, teachers who engage in a constructivist pedagogy 
may actually be more likely as a result to seek out other teachers and become more professionally 
involved. As with other relationships between variables in our dataset, the direction of causality 
between pedagogy and role orientation is difficult to disentangle with a cross-sectional survey. A 
future report will examine the whole set of relationships uncovered and clarify what can be 
concluded from our data and what remains for new research to discover. 
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APPENDIX A: SUPPLEMENTARY TABLES 



TABLE A-1: DISTRIBUTION OF ALL TEACHERS REPORTING ACTIVITY 
BY SUBJECT TAUGHT (PERCENT) 





Monthly Reflective 
Essays 


Weekly Reflective 
Essays 


Monthly Journal Entries 


Weekly Journal Entries 


Subject Taught 


Middle 

Grades 


High 

School 


Middle 

Grades 


High 

School 


Middle 

Grades 


High 

School 


Middle 

Grades 


High 

School 


English 


34 


31 


43 


38 


37 


45 


41 


54 


Social studies 


19 


18 


14 


17 


10 


10 


10 


7 


Science 


15 


22 


11 


23 


15 


9 


14 


11 


Math 


12 


7 


13 


6 


14 


9 


10 


4 


Foreign 

Language 


2 


5 


1 


3 


4 


5 


2 


5 


Computers 


4 


1 


4 


0 


3 


2 


3 


2 


Business Ed. 


1 


4 


0 


1 


0 


2 


0 


3 


Vocational 


1 


3 


2 


3 


2 


8 


2 


7 


Fine Arts 


1 


2 


0 


1 


1 


1 


2 


1 


Total 


100 


100 


100 


100 


100 


100 


100 


100 



Sample: Probability sample. 



TABLE A-2: SUBJECTS WHERE REFLECTIVE WRITING IS MORE COMMON IN MIDDLE SCHOOL 



Percent Reporting Monthly Writing by Students 



Subject & Level* 


Write in Journal 


Essay explaining 
thoughts in length 


Essay or discussion seriously 
assessing own work 


Science (MS) 


34 


44 


24 


Science (HS) 


9 


51 


19 


Math (MS) 


23 


26 


14 


Math (HS) 


11 


21 


6 


Computer (MS) 


24 


47 


31 


Computer (HS) 


9 


13 


12 


Sample: Probability sample. 

*MS = middle school, HS = high school. 



TABLE A-3: REASONS FOR QUESTIONING STUDENTS 
(PERCENT ASKING QUESTION FOR THE FOLLOWING PURPOSES VERY OFTEN OR ALWAYS) 



Subject and Level* 


Ask for the 
correct answer 


Check if students 
did homework 


Have students 
justify or explain 
their reasoning 


Elicit student 
ideas and 
opinion 


Have students 
relate work to their 
own experience 


English (MS) 


52 


29 


74 


75 


70 


English (HS) 


29 


25 


78 


86 


65 


Social Studies (MS) 


53 


28 


66 


57 


49 


Social Studies (HS) 


52 


37 


76 


76 


62 


Science (MS) 


48 


27 


60 


80 


57 


Science (HS) 


48 


31 


63 


75 


51 


Math (MS) 


69 


39 


55 


93 


47 


Math (HS) 


63 


40 


56 


80 


26 



Sample: Probability sample. 

*MS = middle school, HS = high school. 
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TABLE A -4: CORRELATIONS AMONG ELEMENTS OF COGNITIVE CHALLENGE AND 
AMONG ELEMENTS OF ACTIVE LEARNING STRATEGIES, BY TEACHER SUBJECT AND LEVEL 





Correlation Among 
Elements of Cognitive 
Challenge 


Correlation Among Elements of 
Active Learning Strategies 






Reflective 
Writing & 
Deep 
Thinking 


Meaningful 
Thinking & 
Problem- 
Solving 


Projects & 
Group Work 


Projects & 
Infrequent 
Traditional 
Practices 


Groups & 
Infrequent 
Traditional 
Practices 


Mean 

Correlation 
in Row 


English (MS) 


.46 


.64 


.54 


.48 


.36 


.49 


English (HS) 


.49 


.67 


.37 


.52 


.21 


.45 


Social Studies (MS) 


.55 


.69 


.54 


.48 


.50 


.55 


Social Studies (HS) 


.55 


.72 


.35 


.36 


.16 


.43 


Science (MS) 


.52 


.65 


.41 


.46 


.15 


.44 


Science (HS) 


.49 


.58 


.43 


.36 


.10 


.39 


Math (MS) 


.49 


.63 


.45 


.50 


.32 


.48 


Math (HS) 


.49 


.65 


.28 


.50 


-.01 


.38 


Foreign Language (All) 


.39 


.46 


.30 


.33 


.11 


.32 


Computer (All) 


.47 


.49 


.22 


.28 


.02 


.30 


Business (All) 


.40 


.60 


.01 


.43 


.02 


.29 


Vocational (All) 


.47 


.45 


.51 


- .05 


-.35 


.20 


Fine Arts (All) 


.58 


.72 


.28 


.20 


- .10 


.34 


Elementary Self- 
contained 


.55 


.62 


.52 


.49 


.23 


.48 


Elementary Other 


.62 


.57 


.46 


.52 


.23 


.48 


Mean correlation in 
column 


.50 


.61 


.38 


.39 


.13 


.40 



Sample: Probability and purposive samples. 



TABLE A-5: PERCENT OF TEACHERS THAT HAVE STUDENTS WORK IN GROUPS TO COME UP 
WITH A JOINT SOLUTION OR TO APPROACH A PROBLEM OR TASK WEEKLY OR MORE OFTEN 





Elementary 


Middle School 


High School 


All Teachers 


Elementary Self-Contained 


59 






59 


Elementary Other 


49 






49 


English 




32 


37 


34 


Social Studies 




29 


38 


33 


Science 




48 


37 


42 


Math 




59 


45 


52 


Foreign Language 




45 


34 


39 


Computers 




38 


30 


34 


Business 




26 


24 


25 


Vocational 




43 


54 


50 


Fine Arts 




22 


20 


21 


All Teachers 


56 


42 


38 


44 


Sample: Probability sample. 
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TABLE A-6: CORRELATIONS BETWEEN COGNITIVE CHALLENGE AND ACTIVE LEARNING 
STRATEGIES, BY TEACHER SUBJECT AND LEVEL 



Cognitive 





Cognitive 
Challenge & 
Projects 


Cognitive 
Challenge & 
Group Work 


Challenge & 
Infrequent 
Traditional 
Practices 


Cognitive 
Challenge & 
Active 
Learning 


Mean 

Correlation 
in Row 


English (MS) 


.56 


.56 


.39 


.62 


.53 


English (HS) 


.52 


.48 


.32 


.58 


.48 


Social Studies (MS) 


.48 


.54 


.29 


.55 


.46 


Social Studies (HS) 


.38 


.58 


.21 


.55 


.43 


Science (MS) 


.55 


.56 


.17 


.60 


.47 


Science (HS) 


.54 


.57 


.14 


.60 


.46 


Math (MS) 


.50 


.56 


.21 


.59 


.46 


Math (HS) 


.34 


.46 


.07 


.42 


.32 


Foreign Language (All) 


.40 


.23 


.32 


.43 


.34 


Computer (All) 


.38 


.56 


- .07 


.52 


.35 


Business (All) 


.01 


.69 


- .31 


.35 


.19 


Vocational (All) 


.49 


.41 


.11 


.57 


.39 


Fine Arts (All) 


.36 


.52 


.04 


.53 


.36 


Elementary Self-contained 


.64 


.64 


.29 


.70 


.57 


Elementary Other 


.43 


.57 


.12 


.49 


.40 


Mean Correlation in Column 


.44 


.53 


.15 


.54 


.41 



Sample: Probability and purposive samples. 



TABLE A-7: PERCENT OF SECONDARY TEACHERS UNDERTAKING PROJECTS, 

AND OVERALL DISTRIBUTIONS OF THOSE PROJECTS ACROSS SECONDARY SUBJECTS 



% of secondary 
teachers 
undertaking 
week long 
projects, 
monthly 


Distribution of 
weeklong 
projects, by 
secondary 
subjects 
(pie chart view) 


% of secondary 
teachers reporting 
hands on-lab 
activities monthly 


Distribution of 
hands-on/lab 
activities, by 
secondary 
subject 

(pie chart view) 


Distribution of 
teachers by 
subject in the top 
50% overall on 
projects (pie 
chart view) 


English 


62 


26 


51 


14 


17 


Social Studies 


44 


13 


52 


10 


14 


Science 


39 


16 


92 


25 


20 


Math 


11 


5 


53 


15 


16 


Foreign Lang. 


27 


3 


52 


4 


5 


Computers 


73 


7 


99 


6 


4 


Business 


59 


5 


84 


5 


3 


Vocational 


67 


7 


90 


6 


5 


Fine Arts 


92 


8 


91 


5 


5 


All Teachers 


45 


100 


68 


100 


100 



Sample: Probability sample. 
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TABLE A-8: MEAN Z-SCORES ON CONSTRUCTIVIST PRACTICE IN SELECTED CLASS, 

BY SUBJECT AND LEVEL 

Constructivist Practice 



Cognitive Challenge Active Learning 





Meaningful 

Thinking 


Problem- 

Solving 


(Cognitive 

Challenge 

Index) 


Group 

Work 


Projects 


Infrequent 

traditional 

practices 


(Active 

Learning 

Index) 


(Constr- 

uctivist 

Practice 

Index) 


English (MS) 


0.45 


-0.27 


0.27 


-0.11 


-0.14 


0.19 


-0.09 


0.16 


English (HS) 


0.68 


-0.01 


0.53 


0.17 


-0.06 


0.68 


0.25 


0.49 


Social Studies (MS) 


-0.16 


-0.51 


-0.29 


-0.24 


-0.23 


0.11 


-0.25 


-0.29 


Social Studies (HS) 


0.19 


-0.17 


0.09 


0.02 


-0.34 


0.06 


-0.21 


-0.01 


Science (MS) 


-0.03 


0.14 


-0.01 


0.26 


0.09 


0.24 


0.22 


0.10 


Science (HS) 


-0.27 


-0.09 


-0.25 


-0.09 


-0.14 


0.09 


-0.12 


-0.21 


Math (MS) 


-0.32 


0.33 


-0.17 


0.02 


-0.68 


-0.55 


-0.64 


-0.38 


Math (HS) 


-0.66 


0.03 


-0.54 


-0.05 


-0.92 


-0.7 


-0.86 


-0.73 


Foreign Language (All) 


-0.27 


-0.58 


-0.44 


-0.19 


-0.21 


-0.58 


-0.50 


-0.51 


Computer (All) 


- 0.50 


0.02 


-0.36 


-0.29 


0.97 


0.31 


0.56 


0.02 


Business (All) 


-0.71 


-0.78 


-0.82 


-0.76 


0.30 


-0.26 


-0.22 


-0.64 


Vocational (All) 


-0.26 


0.27 


-0.14 


0.15 


1.11 


0.32 


0.80 


0.27 


Fine Arts (All) 


-0.66 


-0.32 


-0.6 


-0.55 


1.33 


0.66 


0.87 


-0.02 


Elem. Self-Contained 


0.45 


0.31 


0.42 


0.18 


0.24 


-0.12 


-0.02 


0.36 


Elementary Other 


0.12 


0.13 


0.13 


0.12 


-0.01 


-0.08 


0.12 


0.09 



Sample: Probability sample. 

*MS = middle school, HS = high school. 
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APPENDIX B: SUMMARY OF STUDY METHODOLOGY 



The Teaching, Learning, and Computing (TLC) study is comprised of completed questionnaire 
responses from teachers, principals, and school technology coordinators from three separate 
samples of schools. Somewhat more than one-half of the 1,616 schools sampled for the study 
(56%) were a stratified national probability sample of elementary (299 schools), middle (253), 
and high schools (346), including 83 private and parochial schools. Those schools were sampled 
with probabilities related to both size (estimated number of full-time teachers, grades 4 to 12) and 
the presence of computer technology (based on an index developed for Quality Education Data, 
Inc.). The sampling universe was the approximately 108,000 schools in the Quality Education 
Data (QED) database. 

The remaining samples of schools are referred to as “purposive samples” and were based on 
compiling, refining, and sampling from lists of two basic types of schools: “High-end Technology 
schools” are schools with substantial amounts of computer technology per capita, including 
schools selected from the QED technology presence index and schools identified through books, 
articles in magazines and school web-sites. “Reform Program schools” were compiled by 
identifying schools or individual teachers who had been long-term (3 year+) participants in one of 
54 different national or regional externally-defined “programs” of major school or instructional 
reform. 

In all three school samples, teachers were sampled from grades 4-12 and from all subjects except 
physical education and special education. At each sampled school, three to five teachers (3, 
elementary; 5, middle and high school) were selected with probabilities related to the teacher’s 
reputed instructional practices and use of technology. A small number of teachers (a maximum of 
2 per school) were selected with certainty (probability equal to 1) based on the principal’s 
attribution of that teacher having an exemplary instructional practice or based on their known 
participation in the selected program of instructional reform. Because unequal probabilities were 
used, at both school and teacher level, all analysis employs weighted data with weights inverse to 
the probability of selection, as modified by stratum-specific non-response rates and within-school 
partial completions of teacher rosters. 

The research began in the Spring of 1997 with a validation study of self-report measures of 
teacher beliefs and practices and exploratory studies of survey measures of changes in teaching 
practices and technology use and school-level investments in technology hardware, software, and 
training and teacher support. The validation study provided self-report data from 72 teachers in 
24 schools and detailed classroom observation and interview data with those same teachers. At 
the school level, pilot versions of surveys were used in order to test measurement approaches for 
studying technology expenditure information, hardware and software acquisition, and 
investments of time and money in teacher training and support activities. 

The data collection itself was the second stage of the project, taking place from January through 
June of 1998, and conducted by the Battelle Centers for Evaluation and Health Research. Data 
collection encompassed an initial district contact information letter, followed by a school mailing, 
in which teachers were rostered and sampled; a subsequent mailing of questionnaires for teachers, 
the school-level technology coordinator, and the principal; and several waves of mail and 
telephone followup, editing, coding, data entry, and data cleaning. The teacher respondents were 
asked to complete a survey booklet about their teaching practice and teaching beliefs that was 21 
pages in length and required approximately 60-75 minutes. Four different versions of the teacher 
survey booklet were used, with overlapping sets of questions. These are called questionnaire 
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versions 1, 2, 3, and 4. The school technology coordinator's booklet was approximately the same 
length as the teacher survey and principally concerned the investments their school has made in 
computer hardware, software, and teacher training and support, measured both financially and in 
units of time, materials, and equipment. The principal's survey booklet was half as long, and 
inquired about technology-related school policies and efforts in school restructuring and reform. 

The third stage of the project involves data analysis, preparation of reports, and the release of 
national data files for secondary analysis. 



PARTICIPATING SCHOOLS 

Across the three samples, 1,215 of the 1,616 schools selected for participation agreed to 
participate in the study (75%). They did so by returning a roster of a specifically requested 
number of teachers (10 in elementary schools; 15 in middle and high schools), providing rough 
estimates of each teacher’s use of computers, projects, and emphasis on critical thinking and 
complex problem-solving. The attained probability sample (rostered schools) consists of 598 
public and 57 private and parochial schools. 

The High-end Technology sample includes 182 rostered schools including 86 entering the sample 
based on having among the highest technology presence index scores in the QED database. The 
remainder were believed to have substantial computer and Internet technology, as identified 
through publicly available information from school Web sites, books, and magazine articles. 

The Reform Program sample includes 378 rostered schools that were identified through various 
sources as being involved in one of 53 different reform efforts. The “reform program” and “high- 
end technology” samples involve some definitional overlap in that 13 of the reform programs 
(with 90 rostered schools) appear to have substantial amounts of technology, while 72 rostered 
high-end technology schools appear to have explicit instructional reform emphases even though 
they did not participate in any of the major reform programs selected. A majority of Reform 
Program schools are involved in a schoolwide reform program (e.g., Coalition of Essential 
Schools, League of Professional Schools, Bay Area School Reform Collaborative, Co-NECT 
Schools) These total 30 separate programs (200 schools) including four with a technology 
emphasis and five that are not ‘programs’ per se but schools linked by a common origin (e.g., 
‘Charter Schools with a constructivist flavor’). In addition, there are four programs that are 
limited to math and/or science (26 schools), 17 programs that enrolled individual teacher 
participants (nine of these are technology-centered), and two programs that recognized individual 
exemplary teachers. 

Lists of participating schools or teachers were obtained directly from the programs in 44 of the 
cases; in the other 9 they were obtained from public sources — lists of participants on World Wide 
Web sites or in books. (In some cases, these were not actually programs — just schools identified 
as exemplary in the public source.) Forty programs provided more schools than were needed so 
that probability sampling was employed to select the particular schools that would be 
incorporated into the study. (In some cases, additional selection criteria were used prior to the 
sampling.) 



SELECTION OF TEACHERS 

At each of the 1,616 studied schools, samples of 3 (elementary) or 5 (middle and high school) 
teachers were drawn through probability sampling methods. A Teacher Roster form was sent to 
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the school principal as the first major mailing to the school (following an introductory letter). 
That form asked the principal to roster either 10 (elementary) or 15 (secondary) teachers of grade 
4 or higher (in some cases limited to the same subject taught by a reform program-participating 
teacher), starting with teachers with last names beginning with a randomly selected letter of the 
alphabet and proceeding alphabetically. The roster form asked for 4 additional pieces of 
information about the rostered teachers that were used to assign sampling weights to each 
rostered teacher (e.g., subject taught, use of computers, use of projects in teaching). 

In addition, two other sources of teachers are incorporated as purposive samples. Approximately 
250 teachers were individually selected from the purposive school samples based on reports 
(public or program-supplied) of their participation in educational reform activities. And finally, 
approximately 800 teachers were chosen through nominations by principals (as part of the Roster 
form) as exemplary practitioners of constructivist approaches to teaching. 



ATTAINED SAMPLE 

Response rates of individually selected teachers, principals, and technology coordinators 
averaged about 70%. This includes 69% of the teachers in the probability sample of schools and 
64% in the purposive schools sample. Altogether, responses were obtained from 4,083 teachers 
of grade 4 and higher in 1,150 schools, as well as 845 technology coordinators and 867 school 
principals. 



QUESTIONNAIRES AND WEIGHTING 

Participating teachers completed 20+ page questionnaires. Four versions of the questionnaire 
were used, with largely overlapping questions, but permitting somewhat greater coverage of 
topics than a single version of equal length would have permitted. The questionnaires dealt with 
five principal topics: teaching philosophy and related beliefs about instruction and assessment; 
teaching practices and strategies followed in the instruction of one class — the class in which the 
teacher felt most satisfied with achieving teaching objectives — ; the ways in which the teacher 
used computers in teaching and professionally and changes over time in the role of computers in 
her teaching practice; changes in her general pedagogical approaches made over the previous 
three years; and a wide range of questions about the teaching environment at the teacher’s school, 
including formal professional development, support for technology, informal interactions with 
other teachers, and pressures experienced in their teaching. In addition, a variety of questions 
about educational and teaching background and current teaching responsibilities were asked at 
various points in the questionnaire. Nearly all questions were of the fixed-response type. Open- 
ended questions about college attended, college major, and current set of courses taught were 
coded into numerical categories. College attended was coded in terms of selectivity, based on 
SAT and ACT scores of admitted freshmen in the year 1983. 

Because unequal probabilities were used, at both school and teacher level, all analysis employs 
weighted data, with weights inverse to the probability of selection, as modified by stratum- 
specific non-response rates and within-school partial completions of teacher rosters. Where 
purposive and national probability samples are combined, weights are adjusted so that the average 
weight for teachers in the purposive schools sample is equal to the average weight for teachers 
from schools in the probability sample. 
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APPENDIX C: DISTRIBUTION FOR INDIVIDUAL TEACHING PHILOSOPHY ITEMS 



TABLE C-1: BELIEFS ABOUT TWO CONTRASTING APPROACHES TO TEACHING 



TRADITIONAL APPROACH 




VS. 


CONSTRUCTIVIST APPROACH 




Ms. Hill was leading her class in an animated way, 


Mr. Jones’ class was also having a discussion, but 


asking questions that the students could answer quickly; 


many of the questions came from the students 


based on the reading they had done the day before. 


themselves. Though Mr. Jones could clarify 


After this review, Ms. Hill taught the class new material, 


students’ questions and suggest where the students 


again using simple questions to keep students attentive 


could find relevant information, he couldn’t really 


and listening to what she said. 




answer most of the questions himself. 






% 


% Tend 




% Tend 






Definitely 


toward % Cannot 


toward 


% Definitely 




Ms. Hill 


Ms. Hill decide 1 


Mr. Jones 


Mr. Jones 


% Total 


Which type of class discussion are you more 
comfortable having in class? 


21 


44 


7 


25 


4 


100 


Which type of discussion do you think most 
students prefer to have? 


20 


34 


10 


28 


9 


100 


From which type of class discussion do you 
think students gain more knowledge? 


14 


30 


15 


29 


13 


100 


From which type of discussion do you think 
students gain more useful skills? 


10 


19 


14 


42 


15 


100 


Sample: Probability sample; questionnaire versions 3 and 4. 












TABLE C-2: BELIEFS ABOUT THE FOLLOWING STATEMENTS 








Strongly 


Moderately 


Slightly 


Slightly 


Moderately 


Strongly 




disagree 


disagree 


disagree 


agree 


agree 


agree 


Teachers know a lot more than students; they 
shouldn't let students muddle around when they 
can just explain the answers directly. 


37 


34 


14 


7 


7 


1 


A quiet classroom is generally needed for 
effective learning. 


15 


29 


15 


20 


16 


6 


Students are not ready for "meaningful" 
learning until they have acquired basic reading 
and math skills. 


23 


18 


14 


13 


18 


16 


It is better when the teacher - not the students - 
decides what activities are to be done. 


7 


20 


20 


23 


22 


8 


Student projects often result in students 
learning all sorts of wrong "knowledge." 


52 


29 


11 


6 


2 


1 


Homework is a good setting for having students 
answer questions posed in their textbooks. 


8 


16 


18 


30 


23 


5 


Students will take more initiative to learn when 
they feel free to move around the room during 
class. 


12 


18 


19 


28 


19 


5 


Students should help establish criteria on which 
their work will be assessed. 


7 


13 


10 


32 


26 


13 


Instruction should be built around problems 
with clear, correct answers, and around ideas 
that most students can grasp quickly. 


14 


29 


19 


18 


13 


6 


How much students learn depends on how 
much background knowledge they have - that 
is why teaching facts is so necessary. 


10 


19 


19 


27 


19 


5 



Sample: Probability sample; items answered in at least 3 of the 4 versions of the questionnaire. 
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TABLE C-3: BELIEFS ABOUT THE FOLLOWING PAIRS OF STATEMENTS 



(PROXIMITY TO THEIR OWN BELIEFS ON A 5 POINT SCALE) 


% Constructivist Perspective 






Vs. 






% Traditional Transmission 
Perspective 


“1 mainly see my role as a facilitator. 1 
try to provide opportunities and 
resources for my students to discover 
or construct concepts for themselves." 


13 


27 


30 


22 


8 


"That's all nice, but students really 
won't learn the subject unless you go 
over the material in a structured way. 
It's my job to explain, to show 
students how to do the work, and to 
assign specific practice." 


"The most important part of instruction 
is that it encourage “sense-making” or 
thinking among students. Content is 
secondary." 


13 


37 


31 


17 


3 


"The most important part of instruction 
is the content of the curriculum. That 
content is the community’s judgment 
about what children need to be able to 
know and do." 


"It is better for students to master a few 
complex ideas and skills well, and to 
learn what deep understanding is all 
about, even if the breadth of their 
knowledge is limited until they are 
older." 


8 


25 


29 


29 


10 


"It is useful for students to become 
familiar with many different ideas and 
skills even if their understanding, for 
now, is limited. Later, in college, 
perhaps, they will learn these things in 
more detail." 


"It is critical for students to become 
interested in doing academic work — 












"While student motivation is certainly 
useful, it should not drive what 


interest and effort are more important 
than the particular subject-matter they 
are working on." 


15 


40 


27 


15 


3 


students study. It is more important 
that students learn the history, 
science, math and language skills in 
their textbooks." 


"It is a good idea to have all sorts of 
activities going on in the classroom. 
Some students might produce a scene 
from a play they read. Others might 
create a miniature version of the set. 
It's hard to get the logistics right, but 
the successes are so much more 
important than the failures." 


20 


28 


26 


20 


7 


"It's more practical to give the whole 
class the same assignment, one that 
has clear directions, and one that can 
be done in short intervals that match 
students' attention spans and the daily 
class schedule." 



Sample: Probability sample. 
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APPENDIX D: DISTRIBUTION FOR INDIVIDUAL TEACHING PRACTICE ITEMS 



TABLE D-1 : IN INTRODUCING THE CURRENT UNIT TO THEIR SELECTED CLASS* (FIRST 
TWO OR THREE LESSONS), PERCENT THAT TOOK THE FOLLOWING APPROACHES 



In introducing the current unit . . . 


Component 


% Did not use 
and rarely do 


% Did not use 
but often do 


% Used this 
time 


% Total 


1 offered the class a reward for doing well 


- 


80 


11 


9 


100 


1 had the students do introductory drills 


T 


35 


20 


45 


100 


Students discussed the topic in small groups 


G 


29 


34 


37 


100 


1 asked students to make conjectures 


M 


21 


28 


51 


100 


1 raised questions about the unit 


M 


52 


26 


22 


100 



Sample: Probability sample. 

‘Class in which teacher most often accomplishes their teaching goals. 
M=Component of Meaningful Thinking Index. 

S=Component of Problem-Solving Index 
G=Component of Group Work Index. 

P= Component of Project Index. 

T=Component of Infrequent Traditional Practices Index. 



TABLE D-2: PERCENT REPORTING THE FOLLOWING ACTIVITIES 
OVER THE LAST 5 HOURS IN THEIR SELECTED CLASS* 







Under 30 


30-60 


1-1/4 to 


Over 2 






Component** 


minutes 


minutes 


2 hours 


hours 


% Total 


Teacher led a whole-class discussion 


T 


33 


34 


21 


12 


100 


Students led a discussion or gave a presentation 
Students worked on their own on assignments at their 


M 


52 


32 


13 


3 


100 


desks 


- 


24 


35 


26 


14 


100 


Students worked together in small groups to 
complete an assignment as a team 


G 


21 


28 


30 


21 


100 



Sample: Probability sample; questionnaire versions 3 and 4. 

‘Class in which teacher most often accomplishes their teaching goals. 
“See Table D-1 for key to abbreviation. 



TABLE D-3: HOW OFTEN STUDENTS IN SELECTED CLASS* 
TAKE PART IN THE FOLLOWING ACTIVITIES 



How often do students... 


Com- 

ponent** 


% 

Never 


% 

Some- 

times 


% 1-3 
times 
per 
month 


% 1-3 
times 
per 
week 


% Almost 
everyday 


% 

Total 


Work individually answering questions 


P 


8 


24 


15 


34 


19 


100 


Do hands-on/laboratory activities 


P 


9 


20 


21 


32 


19 


100 


Work on projects that take a week or more 


P 


10 


42 


28 


8 


11 


100 


Write in a journal 


M 


43 


22 


7 


11 


16 


100 


Suggest or help plan classroom activities or topics 


M 


21 


54 


17 


6 


3 


100 


Work in small groups to come up with a joint solution 


G,S 


6 


27 


24 


30 


14 


100 


Work on problems for which there is no obvious solution *. 


v ; 


; 27- 


. 39 


17 


12 


5 


100 


Write an essay explaining their thoughts in length 


M 


19 


31 


28 


18 


4 


100 



Sample: Probability sample. 

‘Class in which teacher most often accomplishes their teaching goals. 
“See Table D-1 for key to abbreviation. 
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TABLE D-4: HOW OFTEN TEACHERS ARE TRYING TO ACCOMPLISH THE FOLLOWING 
GOALS WHEN ASKING QUESTIONS OF STUDENTS IN THEIR SELECTED CLASS* 



Corn- 

How often do you try to. . . ponent 


% % 

** Never Sometimes 


% % Very % 

Often often Always 


% Total 


See if students know the correct answer T 


1 




18 


28 37 


16 


100 


See if students have done the homework 


10 




30 


27 24 


8 


100 


Elicit students ideas and opinions M 


0 




7 


24 45 


23 


100 


Get students to justify and explain their reasoning M 


1 




6 


18 44 


32 


100 


Have students relate to their own experiences M 


2 




17 


24 38 


20 


100 


Sample: Probability sample. 














*Class in which teacher most often accomplishes their teaching goals. 












**See Table D-1 for key to abbreviation. 














TABLE D-5: HOW OFTEN TEACHERS GIVE THE FOLLOWING TYPES OF ASSIGMENTS 




TO THEIR SELECTED CLASS* 


















% 1-3 


% Once 






Com- 


% 


% Some- times per 


a week 




How often do students . . . ponent** 


Never 


times 


month 


or more 


% Total 


Hold a debate and argue for a point of vie 


M 


44 


44 


10 


3 


100 


Have to design their own problems to solve 


S 


28 


50 


16 


6 


100 


Decide on their own procedures for solving a problem 


S 


16 


45 


26 


13 


100 


Assess their own work on an assignment 


M 


29 


40 


24 


7 


100 


Have to represent the same idea in more than one way 


M 


17 


41 


29 


13 


100 


Make a product that will be used by someone else 


P 


46 


41 


9 


3 


100 


Demonstrate their work to an audience 


P 


53 


35 


9 


2 


100 


Have tasks for which there is no correct answer 


M 


31 


46 


17 


7 


100 



'Class in which teacher most often accomplishes their teaching goals. 
"See Table D-1 for key to abbreviation. 
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